Latitude

Latitude is an open-source, MIT-licensed platform for improving production AI agents. It helps teams capture real agent telemetry, understand recurring failures, and turn them into trackable signals, organized around a workflow of Observe, Understand, and Refine.

In the Observe stage, Latitude collects traces, spans, and sessions covering LLM calls, tool calls, and retrieval steps, with dedicated views for users, tools, memory, and cost. Understand helps teams find what matters via semantic Search, Behaviors clustering, and Signals (recurring failure patterns from scores and annotations), plus Experiments for comparing traffic slices. Refine turns findings into guardrails using Monitors, evaluations, and Datasets that support simulation and regression testing before deployment.

Features

• OTEL compatible: Drop in our SDK or point your existing OpenTelemetry pipeline at Latitude and traces show up.
• Custom signals: Define a signal for any behavior you want to track — a failure mode, policy, or quality check — and Latitude detects it on every matching session.
• Dataset management: Golden datasets built automatically from validated production traces. One per issue, versioned, ready to run evals or regression tests against.
• Human annotations: Leave inline feedback on any trace, span, or output. Your team's judgment becomes structured signal you can search, cluster, and turn into evals.
• Advanced filters: Semantic search, exact text, and metadata filters combined. Find cohorts like "frustrated users on GPT-5.5 in prod after the May release" without writing queries.
• Failure mode clustering: Similar failing traces get grouped into a single issue with examples, trend, affected users, and lifecycle.