MLflow is an open-source platform for AI engineering that covers two areas: LLM applications and agents, and classic machine-learning model development. For agents it traces, evaluates, monitors, and optimizes applications, manages prompts, and routes model calls through a gateway that tracks cost and access. For model training it tracks experiments, evaluates models, keeps a model registry, and deploys models for scoring.
Tracing is built on OpenTelemetry and works with any LLM provider and agent framework, with one-line automatic tracing for more than 60 frameworks. Supported agent frameworks include LangChain, LangGraph, OpenAI Agents SDK, DSPy, Pydantic AI, ADK, CrewAI, LlamaIndex, AutoGen, Agno, Smolagents, Mastra, and the Vercel AI SDK, alongside model providers, gateways such as LiteLLM and OpenRouter, and tools such as Claude Code, OpenCode, and Langfuse. SDKs cover Python, TypeScript and JavaScript, and Java, and MLflow also integrates with MCP.
A setup wizard can hand the instrumentation work to a coding agent: run from a project with Claude Code, Codex, or OpenCode installed, it connects to an MLflow server or Databricks workspace and launches the agent to add tracing to the app. MLflow runs locally, on on-premises clusters, on cloud platforms, or as a managed service, and an Agent Server hosts agents as endpoints with request validation, streaming, and built-in tracing.
Features
- Observability: captures complete traces of LLM applications and agents and monitors quality, cost, and safety in production
- Evaluation: systematic evaluations with 50+ built-in metrics and LLM judges or custom ones, tracked over time to catch regressions
- Prompt management: versions, tests, and deploys prompts with lineage tracking
- Prompt optimization: optimizes prompts automatically with optimization algorithms
- AI Gateway: an OpenAI-compatible API in front of many LLM providers with routing, rate limits, fallbacks, credential management, guardrails, and traffic splitting for A/B tests
- Automatic tracing: one-line tracing for more than 60 agent frameworks, model providers, gateways, and tools
- Coding-agent setup: a wizard that launches Claude Code, Codex, or OpenCode to add tracing to an app
- Experiment tracking: records models, parameters, metrics, and evaluation results across experiments
- Model registry: manages the lifecycle of ML models collaboratively
- Model deployment: batch and real-time scoring on Docker, Kubernetes, Azure ML, AWS SageMaker, and other targets
- Agent Server: hosts agents as endpoints with request validation, streaming, and built-in tracing
- Language support: Python, TypeScript and JavaScript, and Java, with native OpenTelemetry and MCP integration
