A curated list of awesome platforms, tools, practices and resources that helps run LLMs locally
2.4k
Stars
282
Forks
94
Open issues
1
Contributors
AI Analysis
Awesome-local-llm is a curated directory of platforms, tools, models, and resources for running large language models on local hardware without cloud dependencies. It serves developers, researchers, and practitioners who need self-hosted LLM infrastructure, from inference engines like Ollama and llama.cpp to UI clients, fine-tuning frameworks, and evaluation tools. This is a reference collection best suited for practitioners actively building local LLM systems, not a library or framework for ...
Inferred from signals mentioned in the README (tests, CI, type safety) — not a review of the actual code.
AI's overall editorial judgment — not an average of the bars above, can weigh other factors too.
Curated index of local LLM platforms and tools, not an implementation
awesome-local-llm is a curated list repository documenting platforms, inference engines, tools, and resources for running large language models locally rather than via cloud APIs. It serves as a discovery and reference resource for developers, researchers, and practitioners exploring the local LLM ecosystem. Created June 2025, it has grown to ~2,400 stars in one year, gaining ~34 stars per week as of mid-2026, indicating steady interest in the local-LLM movement rather than viral adoption.
Launched June 2025 during a period of accelerating interest in local model inference (ollama, llama.cpp, vLLM gaining traction). Follows the Awesome list format, a GitHub convention for community-curated knowledge collections. Positioned within a larger ecosystem of awesome-LLM repositories but focuses specifically on the 'local inference' niche rather than general LLM resources.
Steady organic growth from ~400 stars in first month to 2,336 by June 2026 suggests sustained but not explosive adoption. Growth likely driven by: (1) maturation of local inference tooling (ollama, llama.cpp); (2) enterprise and privacy-conscious demand for on-premises LLM deployment; (3) developer interest in cost control and offline-capable AI systems. Recent activity (last push 2026-06-23) indicates active curation. Linear growth pattern suggests a niche consolidating rather than trending.
Adoption not verified through direct usage metrics. Evidence is indirect: (1) 2,336 GitHub stars indicates discovery and interest; (2) 267 forks suggest reuse by teams for internal reference or derivative lists; (3) consistent weekly star growth (~34/week) implies sustained organic reach rather than one-time viral spike; (4) presence in similar-repos query alongside 115k-star awesome-llm-apps suggests it occupies a recognized subcategory. No data on enterprise adoption, corporate citations, or institutional use.
This is a curation and documentation repository, not an implementation. It organizes links, descriptions, and star badges across categories: inference platforms (LM Studio, LocalAI, jan), inference engines (ollama, llama.cpp, vLLM), UIs, models, tools (agents, RAG, fine-tuning), hardware, tutorials, and communities. Architecture is informational and metadata-driven, not algorithmic.
Not applicable. This is a curated list, not a library or application. No code to test.
Active curation: last commit 2026-06-23 (6 days before analysis date). Repository created 2025-06-03, suggesting ~12 months of operation. 267 forks and 2,336 stars indicate community engagement and reuse. No explicit data on issue response rate or contributor diversity from metadata alone. Steady push activity suggests human curation rather than automated updates.
ADOPT IF: you are building a knowledge base, comparison matrix, or team reference for local LLM deployment; you need a starting point for exploring inference platforms, fine-tuning frameworks, or privacy-preserving model serving. AVOID IF: you need runnable code, libraries, or implementations—this is a curation index, not infrastructure. You should use the *tools listed* (ollama, llama.cpp, vLLM), not the list itself, for production work. MONITOR IF: you are tracking the maturation of the local-LLM ecosystem; the list's scope and growth may signal whether this remains a niche or becomes mainstream for enterprise deployments.
Independent dimensions
Mainstream potential
3/10
Technical importance
5/10
Adoption evidence
4/10
- Curation accuracy and timeliness not verified—some links may be outdated, abandoned, or mischaracterized; no systematic validation process evident.
- Link rot over time: as projects mature or disappear, list entries may become stale without active maintenance discipline.
- Scope creep: as local LLM tooling expands, the list risks becoming too broad or duplicative with general awesome-LLM lists.
- Single maintainer dependency: no evidence of multiple active maintainers; project may slow if primary curator loses interest.
- No versioning or archival strategy stated—unclear how breaking changes (e.g., deprecated tools) are handled or communicated to users relying on the list.
Likely to remain a stable, focused discovery resource for the local-LLM practitioner community. Growth may plateau as the category matures and tools consolidate (e.g., ollama dominance). Could become a reference point in academic or industry reports on edge AI deployment. Unlikely to reach the scale of general awesome-LLM lists, but may establish itself as the canonical index for local inference specifically.
Newsletter
Get analyses like this every Monday
Free weekly digest of the most interesting open-source discoveries.
Languages
No language breakdown available.
Information
- License
- MIT
- Last updated
- 6d ago
- Created
- 13mo ago
- Analyzed with
- anthropic/claude-haiku-4-5
Stars over time
Contributors over time
Top 100 contributors only — repos with more will plateau at 100.
Open issues
No open issues — clean slate.
Top contributors
Recent releases
No releases published yet.
Similar repos
Hannibal046/Awesome-LLM
Awesome-LLM is a curated index of Large Language Model resources, including...
WangRongsheng/awesome-LLM-resources
This is a curated index of Large Language Model resources, covering multimodal...
corca-ai/awesome-llm-security
This is a curated collection (awesome-list) of research papers, tools,...
| Repository | Stars | Week Δ | Language | Score | Updated |
|---|---|---|---|---|---|
|
rafska/awesome-local-llm
this
|
2.4k | +29 | — | 7/10 | 6d ago |
|
Hannibal046/Awesome-LLM
|
27.1k | — | — | 7/10 | 11mo ago |
|
WangRongsheng/awesome-LLM-resources
|
8.7k | — | — | 7/10 | 9h ago |
 jamesob/local-llm
|
1.3k | — | Shell | 8/10 | 2d ago |
|
corca-ai/awesome-llm-security
|
1.6k | — | — | 7/10 | 11mo ago |
 Troyanovsky/Local-LLM-Comparison-Colab-UI
|
1.1k | — | Jupyter Notebook | 6/10 | 6mo ago |
27k stars, broader LLM resource scope (not local-specific). awesome-local-llm is narrower but more focused on inference-on-edge and privacy-preserving deployment; Awesome-LLM covers general LLM landscape.
8.6k stars, general LLM resources. Less specific to local inference; awesome-local-llm provides more curated tooling for running models on personal/edge hardware.
1.6k stars, security-focused niche. Both are narrower than general LLM lists; awesome-local-llm and awesome-llm-security serve complementary but distinct sub-communities.
3.4k stars, specialized to coding LLMs. Similar scale and niche depth to awesome-local-llm; both avoid competing with 115k-star awesome-llm-apps by serving specific verticals.
115k stars, applications built with LLMs rather than inference tooling. Different audience: awesome-llm-apps targets app builders; awesome-local-llm targets infrastructure and deployment practitioners.
[unsloth](https://github.com/unslothai/unsloth) - unified web UI for training and running open models like Qwen, DeepSeek, and Gemma locally
[LocalAI](https://github.com/mudler/LocalAI) - the free, open-source alternative to OpenAI, Claude and others
[jan](https://github.com/menloresearch/jan) - an open source alternative to ChatGPT that runs 100% offline on your computer
[ChatBox](https://github.com/ChatBoxAI/ChatBox) - user-friendly desktop client app for AI models/LLMs
[lemonade](https://github.com/lemonade-sdk/lemonade) - a local LLM server with GPU and NPU Acceleration
[ollama](https://github.com/ollama/ollama) - get up and running with LLMs
[llama.cpp](https://github.com/ggml-org/llama.cpp) - LLM inference in C/C++
[vllm](https://github.com/vllm-project/vllm) - a high-throughput and memory-efficient inference and serving engine for LLMs
[exo](https://github.com/exo-explore/exo) - run your own AI cluster at home with everyday devices
[BitNet](https://github.com/microsoft/BitNet) - official inference framework for 1-bit LLMs
[sglang](https://github.com/sgl-project/sglang) - a fast serving framework for large language models and vision language models
[TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) - provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs
[Nano-vLLM](https://github.com/GeeeekExplorer/nano-vllm) - a lightweight vLLM implementation built from scratch
[omlx](https://github.com/jundot/omlx) - LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
[koboldcpp](https://github.com/LostRuins/koboldcpp) - run GGUF models easily with a KoboldAI UI
[mistral.rs](https://github.com/EricLBuehler/mistral.rs) - fast, flexible LLM inference
[dynamo](https://github.com/ai-dynamo/dynamo) - a datacenter scale distributed inference serving framework
[flashinfer](https://github.com/flashinfer-ai/flashinfer) - kernel library for LLM serving
[mlx-lm](https://github.com/ml-explore/mlx-lm) - generate text and fine-tune large language models on Apple silicon with MLX
[gpustack](https://github.com/gpustack/gpustack) - simple, scalable AI model deployment on GPU clusters
[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) - Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices
[mlx-vlm](https://github.com/Blaizzy/mlx-vlm) - a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX
[executorch](https://github.com/pytorch/executorch) - on-device AI across mobile, embedded and edge for PyTorch
[mini-sglang](https://github.com/sgl-project/mini-sglang) - a lightweight yet high-performance inference framework for Large Language Models
[distributed-llama](https://github.com/b4rtaz/distributed-llama) - connect home devices into a powerful cluster to accelerate LLM inference
[LiteRT](https://github.com/google-ai-edge/litert) - Google's on-device framework for high-performance ML & GenAI deployment on edge platforms, via efficient conversion, runtime, and optimization
[ik_llama.cpp](https://github.com/ikawrakow/ik_llama.cpp) - llama.cpp fork with additional SOTA quants and improved performance
[aphrodite-engine](https://github.com/aphrodite-engine/aphrodite-engine) - large-scale LLM inference engine
[FastFlowLM](https://github.com/FastFlowLM/FastFlowLM) - run LLMs on AMD Ryzen™ AI NPUs
[tokenspeed](https://github.com/lightseekorg/tokenspeed) - a speed-of-light LLM inference engine
[krasis](https://github.com/brontoguana/krasis) - a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware
[vllm-gfx906](https://github.com/nlzy/vllm-gfx906) - vLLM for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60
[llm-scaler](https://github.com/intel/llm-scaler) - run LLMs on Intel Arc™ Pro B60 GPUs
[Open WebUI](https://github.com/open-webui/open-webui) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
[Lobe Chat](https://github.com/lobehub/lobe-chat) - an open-source, modern design AI chat framework
[Text generation web UI](https://github.com/oobabooga/text-generation-webui) - LLM UI with advanced features, easy setup, and multiple backend support
[SillyTavern](https://github.com/SillyTavern/SillyTavern) - LLM Frontend for Power Users
[Page Assist](https://github.com/n4ze3m/page-assist) - Use your locally running AI models to assist you in your web browsing
[BullshitBench](https://github.com/petergpt/bullshit-benchmark) - measure whether AI models challenge nonsensical prompts instead of confidently answering them
[vakra](https://github.com/IBM/vakra) - a benchmark for evaluating multi-hop, multi-source tool-calling in AI agents
[Mistral AI](https://huggingface.co/mistralai) - a pioneering French artificial intelligence startup