Evals

Harbor is a framework for evaluating, benchmarking, and optimizing AI agents and language models across various environments and cloud providers. It...

Future AGI is an open-source platform for evaluating, tracing, and improving LLM and AI agent applications, combining evals, observability,...

A continuously updated reference guide for designing production AI systems, covering RAG architectures, LLM engineering, agentic AI, and AI...

Phoenix is an AI observability and evaluation platform built for monitoring, evaluating, and debugging LLM applications and AI agents. It serves data...

Mastra is a TypeScript framework for building AI-powered applications and agents, positioning itself as a bridge between prototyping and production....

MCPJam Inspector is a testing and evaluation platform designed specifically for debugging, inspecting, and evaluating MCP (Model Context Protocol)...

Laminar is an open-source observability platform purpose-built for AI agents, offering tracing, signals, evaluations, dashboards, and data annotation...

Pydantic Logfire is an observability platform designed specifically for production LLM and agent systems, built by the Pydantic team. It provides...

AgentOps is a Python SDK for monitoring, observability, and cost tracking of AI agents across multiple frameworks (CrewAI, Langchain, Autogen, OpenAI...

Kiln is a comprehensive workbench for the full AI development lifecycle, combining evaluations, prompt optimization, RAG, fine-tuning, synthetic data...

RAGLite is a Python toolkit for building Retrieval-Augmented Generation (RAG) systems with pluggable LLM providers, vector databases (DuckDB or...

TruLens is a specialized evaluation and tracking platform for LLM applications and AI agents, designed to systematically assess performance through...

Simba is an open-source, production-ready customer service assistant platform built with TypeScript that emphasizes evaluation metrics and...

Evalite is a TypeScript-native, local-first tool for evaluating and testing LLM-powered applications. It serves developers building AI-driven...