Clear ×
★ –

Inference engine

5 repositories found

+ Follow domain
xLLM-AI

xLLM-AI

xllm
C++ Rising Star AI analysis

xLLM is a high-performance inference engine optimized for LLM, VLM, and other AI models, specifically designed for Chinese AI accelerators and...

Production ready Niche use case
deepseek glm inference
1.5k
256
172
89/wk
active
7/10 AI & ML
qualcomm

qualcomm

ai-hub-models
Python AI analysis

Qualcomm AI Hub Models is a curated collection of pre-optimized machine learning models designed specifically for deployment on Qualcomm devices...

Production ready Niche use case
deeplearning demos inference
1.2k
207
11
11/wk
active
8/10 AI & ML
nobodywho-ooo

nobodywho-ooo

nobodywho
Rust AI analysis

NobodyWho is a multi-platform inference engine written in Rust that enables running LLMs locally and offline across Kotlin, Swift, Python, Flutter,...

Production ready Beginner friendly Niche use case
ai flutter godot
1k
70
36
8/wk
active
8/10 AI & ML
dphnAI

dphnAI

sonar
C++ AI analysis

Sonar is a high-performance LLM inference engine optimized for serving HuggingFace-compatible models at scale, built on vLLM's Paged Attention...

Production ready Niche use case
api-rest cuda inference-engine
1.8k
202
127
8/wk
active
7/10 AI & ML
ovg-project

ovg-project

kvcached
Python AI analysis

kvcached is a KV cache management library for LLM serving and training on shared GPUs, using OS-style virtual memory abstraction to enable elastic,...

Production ready Niche use case
elastic-kvcache gpu-mutiplexing gpu-sharing
1.1k
122
90
3/wk
recent
8/10 AI & ML