Phoenix is an AI observability platform for experimenting with, evaluating, and troubleshooting LLM applications. It traces application runtime with OpenTelemetry-based instrumentation, runs LLM-based evaluations over responses and retrievals, and keeps versioned datasets for experiments. It is vendor- and language-agnostic, with instrumentation for frameworks including the OpenAI Agents SDK, Claude Agent SDK, LangGraph, Vercel AI SDK, and Mastra, and it runs locally, in a container, or in the cloud.
You start a server with phoenix serve and send traces from your application. A CLI setup command detects your framework and LLM provider, installs the matching OpenInference instrumentation, and configures trace export; it can be driven by a coding agent such as Claude Code, Codex, or Cursor. Container images and a Helm chart cover Docker and Kubernetes deployments, with templates for Railway, Render, Google Cloud, Azure, and AWS.
The main arize-phoenix package contains the whole platform. Smaller companion packages can be used against a deployed instance: arize-phoenix-otel wraps OpenTelemetry primitives with Phoenix-aware defaults, arize-phoenix-client talks to the server over its OpenAPI REST interface, and arize-phoenix-evals provides evaluators for RAG relevance, answer relevance, and more. TypeScript packages, plus JavaScript, Java, and Go integrations, extend coverage beyond Python.
Features
- Tracing: OpenTelemetry-based runtime traces of LLM calls, retrievals, and tool use
- Evaluation: LLM-judged response and retrieval evals to benchmark application behaviour
- Datasets: versioned example sets for experimentation, evaluation, and fine-tuning
- Experiments: track and score changes to prompts, models, and retrieval
- Playground: tune prompts, compare models, adjust parameters, and replay traced LLM calls
- Prompt management: version control, tagging, and experimentation for prompt changes
- PXI (Phoenix Intelligence): a built-in AI engineering agent for debugging traces, iterating on prompts, and navigating the product
- Remote MCP server: an /mcp endpoint that lets Claude Code, Cursor, and other MCP clients query traces, datasets, and experiments
- Agent-assisted setup: npx @arizeai/phoenix-cli setup or px setup instruments a project automatically
- Deployment: Docker Hub images, a Helm chart, and one-click templates for major cloud providers
- Companion packages: arize-phoenix-otel, arize-phoenix-client, arize-phoenix-evals, and TypeScript equivalents
Integrates with
AutoGen
Microsoft multi-agent framework now in maintenance mode, succeeded by Microsoft Agent Framework
CrewAI
Python multi-agent framework pairing autonomous role-based Crews with event-driven Flows for precise orchestration
DSPy
Compose LM pipelines as declarative Python modules, then optimize their prompts and weights algorithmically
Guardrails
Python framework that adds input/output validation guards to LLM applications and generates structured data
