OpenLIT

OpenLIT is a self-hosted platform for AI engineering that combines OpenTelemetry-native observability for LLM applications with evaluations, a rule engine, prompt management, a secrets vault, and a model playground. A single openlit.init() call in Python (with TypeScript and Go SDKs also available) auto-instruments more than 50 LLM providers, agent frameworks, and vector databases, and reports GPU metrics; traces and metrics follow the OpenTelemetry GenAI semantic conventions, so they can go to OpenLIT's own dashboard or to any OTLP-compatible backend.

The stack deploys with Docker Compose or a Helm chart: the SDK sends data to an OpenTelemetry Collector, which stores it in ClickHouse, and the OpenLIT web UI reads from ClickHouse. When no OTLP endpoint is configured, the SDK prints traces to the console for development.

A separate CLI extends the same tracing to local coding agents. openlit coding install adds hooks to Claude Code, Cursor, or Codex that emit traces for each session, prompt, tool call, file edit, subagent spawn, and code-impact event, using genai.* conventions plus a codingagent.* extension namespace, and the dashboard shows sessions, prompts, costs, and per-vendor breakdowns.

Features

• Analytics dashboard: application and GPU health, request metrics, costs, and user interactions
• OpenTelemetry SDKs: vendor-neutral Python, TypeScript, and Go SDKs that export to existing observability tools
• Evaluations: 11 LLM-as-a-judge types covering hallucination, bias, toxicity, safety, instruction following, completeness, conciseness, sensitivity, relevance, coherence, and faithfulness, with provided context treated as the source of truth
• Rule engine: AND/OR conditions on runtime trace attributes that select contexts, prompts, and evaluation configs dynamically
• Cost tracking: custom pricing files for fine-tuned or custom models
• Exceptions dashboard: tracks recurring errors and exceptions
• Prompt Hub: versioned prompt management shared across applications
• Vault: central storage for API keys and secrets
• OpenGround: compare multiple LLMs side by side
• Coding-agent tracing: CLI hooks for Claude Code, Cursor, and Codex with an openlit doctor diagnostic
• Integrations: 50+ auto-instrumented LLM providers, frameworks, and vector databases, plus NVIDIA and AMD GPU collectors