MLflow

MLflow is an open-source platform for managing the lifecycle of AI applications, covering both LLM-based agents and classical machine learning models. For agents and LLM applications it provides tracing, evaluation, prompt management and optimisation, and an AI Gateway; for model training it provides experiment tracking, model evaluation, a model registry, and deployment to batch and real-time scoring targets.

Tracing is built on OpenTelemetry and captures complete traces of an application's calls, with one-line automatic instrumentation for more than 60 frameworks and providers. A typical setup starts a tracking server with uvx mlflow server, calls an autolog function such as mlflow.openai.autolog(), and then inspects traces and metrics in the web UI; mlflow agent setup installs MLflow skills and launches a coding agent to add tracing to an existing app. MLflow supports Python, TypeScript/JavaScript, Java, and other languages, and integrates natively with OpenTelemetry and MCP.

Features

• Observability: OpenTelemetry-based traces for any LLM provider or agent framework, with production monitoring of quality, cost, and safety
• Evaluation: systematic evaluations with 50+ built-in metrics and LLM judges, or custom definitions, tracked over time to catch regressions
• Prompt registry: version, test, and deploy prompts with lineage tracking
• Prompt optimisation: automated algorithms that improve prompt performance
• AI Gateway: an OpenAI-compatible gateway that routes requests, applies rate limits and fallbacks, manages credentials, adds guardrails, and splits traffic for A/B tests
• Experiment tracking: log models, parameters, metrics, and evaluation results across runs
• Model registry: collaborative lifecycle management for trained models
• Deployment: batch and real-time serving on Docker, Kubernetes, Azure ML, AWS SageMaker, and more
• Integrations: one-line auto-tracing for 60+ frameworks, and SDKs for Python, TypeScript/JavaScript, and Java