Rocm

InferenceX is an open-source continuous inference benchmarking research platform that tracks real-time performance of LLM inference frameworks across...

LMCache is a KV cache management layer that optimizes Large Language Model inference by providing efficient memory management across multiple...

Lemonade is a local AI server that runs optimized LLMs on user GPUs and NPUs (particularly AMD Ryzen AI and Radeon), providing OpenAI-compatible APIs...

Chatterbox TTS Server is a self-hosted text-to-speech API server that wraps Resemble AI's open-source Chatterbox TTS model, offering...

CuPy is a GPU-accelerated array library that provides NumPy and SciPy compatibility for NVIDIA CUDA and AMD ROCm platforms, enabling existing...

GPUStack is an open-source GPU cluster manager designed for high-performance AI model serving (vLLM, SGLang, TensorRT-LLM) and on-demand GPU instance...

Sonar is a high-performance LLM inference engine optimized for serving HuggingFace-compatible models at scale, built on vLLM's Paged Attention...

Apache TVM is an open machine learning compiler framework that optimizes and deploys deep learning models across diverse hardware backends (GPUs,...

IREE is an MLIR-based compiler and runtime that lowers machine learning models to a unified intermediate representation, scaling from datacenter...

Burn is a Rust-based tensor library and deep learning framework that unifies training and inference through a single codebase, eliminating the...

DeePMD-kit is a specialized deep learning package for constructing machine learning-based interatomic potential models and performing molecular...

stdgpu is a C++17 library providing STL-like GPU data structures (atomic, bitset, deque, queue, vector, hash table, etc.) optimized for CUDA, OpenMP,...