Opik

Opik is an open-source platform for tracing, evaluating, and monitoring applications built on large language models. It covers the lifecycle from a first development trace through production monitoring: deep tracing of model calls and agent activity as full trace trees, datasets and experiments for evaluation, prompt management and a playground, dashboards and online evaluation rules for production, guardrails, and an agent-optimizer SDK for improving prompts and agents.

Instrumenting an application is one decorator. Install the Python SDK, run opik configure, and wrap a function with @track; every call is logged with its nested calls, so multi-step agents and pipelines are captured rather than single model calls. A TypeScript SDK and a REST API cover other entry points, and first-party OpenTelemetry support means any language with an OpenTelemetry SDK, including Java, Ruby, and .NET, can send traces.

Evaluation runs on datasets and experiments with both code-based metrics and model-as-a-judge metrics for hallucination detection, moderation, and retrieval-augmented-generation assessment such as answer relevance and context precision. A PyTest integration puts those checks in a continuous-integration pipeline, and online evaluation rules apply the same judges to production traffic. Feedback scores can be annotated from the SDK or the interface.

The whole platform is Apache-2.0 licensed and self-hostable, including the server backend and web application rather than only a client SDK. Deployment options are a hosted cloud account, a local Docker Compose install driven by an opik.sh or opik.ps1 script with infrastructure-only, backend, guardrails, build, verify, stop, and clean profiles, or a Kubernetes install through a Helm chart. The project positions itself against LangSmith, Arize Phoenix and Arize AX, Weights & Biases Weave, Langfuse, and Braintrust, and states that it is framework-agnostic and designed for high trace volume.

Features

• Agent tracing: full trace trees covering model calls, tool executions, retrieval steps, and conversation logs
• One-line instrumentation: the @track decorator logs a function and everything it calls
• Evaluation: datasets, experiments, code metrics, and model-as-a-judge metrics for hallucination, moderation, and RAG quality
• CI checks: a PyTest integration runs evaluations on every commit
• Production monitoring: dashboards for feedback scores, trace counts, and token usage, plus online evaluation rules
• Prompt management: versioned prompts and a playground for comparing prompts and models
• Agent optimizer: an SDK for systematically improving prompts and agents
• Guardrails: optional guardrail services enabled through the installation script
• SDKs and OpenTelemetry: Python and TypeScript clients, a REST API, and first-party OpenTelemetry ingestion
• Framework integrations: roughly sixty direct integrations including LangChain, LangGraph, LlamaIndex, CrewAI, Haystack, DSPy, Dify, LiteLLM, Ollama, and Vercel AI SDK
• Agent-client integrations: Claude Code, Cursor, OpenClaw, and OpenWebUI sessions can be logged, and an MCP server drives Opik from those clients
• Self-hosting: Apache-2.0 for the full platform, deployable with Docker Compose locally or Helm on Kubernetes