Sglang

InferenceX is an open-source continuous inference benchmarking research platform that tracks real-time performance of LLM inference frameworks across...

Local Studio is a desktop and web control panel for running self-hosted large language models locally via vLLM, SGLang, llama.cpp, and MLX backends....

Mooncake is a specialized KVCache-disaggregated serving platform for large language model inference, designed to handle high-throughput request...

AutoRound is a specialized quantization toolkit for Large Language Models and Vision-Language Models, designed to achieve high accuracy at ultra-low...

GPUStack is an open-source GPU cluster manager designed for high-performance AI model serving (vLLM, SGLang, TensorRT-LLM) and on-demand GPU instance...

GPTQModel is a specialized LLM quantization and compression toolkit providing hardware-accelerated inference across NVIDIA, AMD, Intel, Huawei, and...

Dynamo is an open-source, datacenter-scale orchestration layer for distributed LLM inference that coordinates multiple inference engines (vLLM,...

OpenClaw-RL is a reinforcement learning framework that enables training personalized agents through natural language interaction, supporting...

kvcached is a KV cache management library for LLM serving and training on shared GPUs, using OS-style virtual memory abstraction to enable elastic,...

MOVA is an open-source foundation model for synchronized video–audio generation that produces video and audio in a single inference pass rather than...