Llamacpp

Atomic Chat is a local-first desktop and mobile application that runs open-weight LLMs privately on users' machines with an OpenAI-compatible API...

Local Studio is a desktop and web control panel for running self-hosted large language models locally via vLLM, SGLang, llama.cpp, and MLX backends....

llama-swap is a Go-based proxy server that enables hot-swapping between multiple local AI models running on OpenAI/Anthropic-compatible inference...

Wllama is a WebAssembly binding for llama.cpp that enables large language model inference directly in web browsers without requiring a backend...

Cactus is a low-latency AI inference engine purpose-built for mobile devices and wearables, enabling on-device LLM, vision, and speech processing via...

Paddler is an open-source LLM/VLM load balancer and serving platform designed for self-hosting inference at scale on private infrastructure,...

RunAnywhere is a production-ready SDK for running AI models (LLMs, vision, speech) entirely on-device across iOS, Android, Flutter, React Native, and...

Shimmy is a pure-Rust, single-binary WebGPU inference engine that provides OpenAI-compatible API endpoints for running GGUF models locally without...

RamaLama is a containerized AI model serving tool that abstracts away GPU configuration complexity by automatically detecting hardware and pulling...

Khoj is a self-hosted AI assistant platform that integrates local and cloud LLMs with semantic search over user documents and web content. It serves...

Jan is a desktop application that enables users to run large language models (LLMs) locally on their computers with full privacy and control,...

LLamaSharp is a C#/.NET library for running LLaMA and LLaVA language models locally on consumer hardware, built as a wrapper around llama.cpp. It...

gptme is a terminal-based AI agent that integrates with local tools (code execution, shell commands, web browsing, vision) and supports multiple LLM...

Kalosm is a Rust ecosystem for running local, pre-trained AI models (LLMs, audio transcription, image segmentation, embeddings) with optional GPU...

Xinference is a production-ready inference server that lets developers swap between different LLMs (open-source, speech, and multimodal models)...

AGiXT is a Python-based AI agent automation platform that orchestrates instruction management and task execution across multiple AI providers...

Spark NLP is a distributed natural language processing library built on Apache Spark, offering 100,000+ pretrained models and pipelines across 200+...

LlamaChat is a native macOS application that enables local inference of LLaMA, Alpaca, and GPT4All models directly on a user's Mac without cloud...

OllamaSharp is a .NET library that provides bindings for the Ollama API, enabling developers to run and interact with large language models locally...

llmware is a unified Python framework for building enterprise RAG (retrieval-augmented-generation) pipelines using small, specialized models...