Langfuse

Langfuse is an LLM engineering platform for teams that develop, monitor, evaluate, and debug AI applications. An application instruments its code with the SDKs or an automated integration and sends traces to Langfuse, which records LLM calls together with related logic such as retrieval, embedding, and agent actions, so a team can inspect and debug logs and user sessions. It runs as a managed cloud service or self-hosted on your own infrastructure (a single machine with Docker Compose, a VM, or Kubernetes with Helm, with Terraform templates for AWS, Azure, and GCP). Storage uses the ClickHouse database.

Beyond tracing, it manages prompts centrally with version control and server- and client-side caching, so prompt iteration adds no latency to the application. Evaluation covers LLM-as-a-judge, code evaluators, user feedback collection, manual labeling, and custom evaluation workflows through the API and SDKs. Datasets hold test sets and benchmarks for pre-deployment testing and structured experiments, and a playground lets you jump from a bad trace to iterating on the prompt and model configuration.

Instrumentation comes in several forms: Python and JS/TS SDKs for manual tracing, a drop-in replacement for the OpenAI SDK, callback handlers for LangChain, LlamaIndex, and Haystack, the LiteLLM proxy, the Vercel AI SDK, and Mastra. Further integrations cover Instructor, DSPy, Mirascope, Ollama, Amazon Bedrock, AutoGen, Flowise, Langflow, Dify, Open WebUI, Promptfoo, LobeChat, Vapi, Inferable, Gradio, Goose, smolagents, and CrewAI. A public API with an OpenAPI spec, a Postman collection, and typed SDKs lets teams build their own LLMOps workflows on the same building blocks.

Self-hosted instances report basic usage statistics to a central server by default. The telemetry excludes raw traces, prompts, observations, scores, and dataset contents, and the open-source edition can opt out.