Llmops

vLLM Ascend Plugin is a community-maintained hardware acceleration plugin for vLLM that enables large language model serving on Huawei Ascend...

Bifrost is a high-performance AI gateway that unifies access to 23+ LLM providers through a single OpenAI-compatible API, with features including...

RTP-LLM is Alibaba's high-performance LLM inference engine designed for production-scale serving of large language models across multiple business...

VectorChord is a PostgreSQL extension for scalable, cost-effective vector search that compresses and stores vectors with minimal storage overhead...

Langfuse is an open-source AI engineering platform that provides LLM observability, evaluation, metrics, prompt management, and a playground...

Plano is an AI-native proxy and data plane built on Envoy that centralizes orchestration, safety, observability, and intelligent LLM routing for...

LangWatch is a platform for evaluating and testing LLM-powered agents end-to-end, combining tracing, datasets, evaluation, and prompt optimization in...

LiteLLM is an open-source AI Gateway and Python SDK that provides a unified OpenAI-compatible interface to 100+ LLM providers (OpenAI, Anthropic,...

Opik is an open-source observability and evaluation platform specifically designed for LLM applications, RAG systems, and agentic workflows. It...

This is the official GitHub repository for the LLM Engineer's Handbook, a practical guide covering the full lifecycle of building and deploying LLM...

OpenInference is a set of OpenTelemetry conventions and instrumentation plugins that enable comprehensive tracing and observability for AI...

Ragas is a Python toolkit for evaluating and optimizing Large Language Model applications through objective metrics, test data generation, and...

Phoenix is an AI observability and evaluation platform built for monitoring, evaluating, and debugging LLM applications and AI agents. It serves data...

Promptfoo is a CLI and library for evaluating, testing, and red-teaming LLM applications, enabling users to compare model performance across multiple...

Rig is a Rust library for building modular, scalable LLM-powered applications with support for 20+ model providers and 10+ vector stores under...

CubeStudio is a comprehensive cloud-native MLOps/AI platform designed for enterprise machine learning workflows, supporting distributed training,...

LLM Guard is a Python security toolkit for protecting Large Language Model interactions through input sanitization, harmful content detection, data...

OpenLIT is an open-source observability platform designed for AI engineering, providing OpenTelemetry-native monitoring for LLMs, vector databases,...

A structured educational course teaching how to build production-ready AI assistants combining LLMs, RAG (Retrieval-Augmented Generation), and...

Helicone is an open-source AI Gateway and LLM observability platform that enables developers to monitor, evaluate, and manage large language model...

Laminar is an open-source observability platform purpose-built for AI agents, offering tracing, signals, evaluations, dashboards, and data annotation...

Paddler is an open-source LLM/VLM load balancer and serving platform designed for self-hosting inference at scale on private infrastructure,...

Agenta is an open-source LLMOps platform designed for building and managing reliable LLM applications through prompt management, evaluation, and...

MLflow is a comprehensive open-source platform for managing the lifecycle of AI applications, specifically optimized for LLM agents, generative AI,...

Coze Loop is a full-lifecycle AI agent development and operations platform offering prompt engineering, evaluation, and observability capabilities....

Portkey AI Gateway is a TypeScript-based routing layer that abstracts access to 1,600+ language, vision, audio, and image models across multiple...

Agently is a Python framework for building AI application backends with structured outputs, observable actions, runtime skills, MCP capabilities, and...

MCP Router is a desktop application for centralized management of Model Context Protocol (MCP) servers across Windows and macOS. It serves developers...

Giskard is an open-source Python library for testing, evaluating, and red-teaming agentic AI systems. It specializes in dynamic multi-turn agent...

OpenLLMetry is an open-source observability framework built on OpenTelemetry that provides instrumentation for LLM applications, vector databases,...