PandaProbe is an agent engineering platform for tracing, evaluating, monitoring, and debugging AI agents. Teams instrument their agents with a Python client library, send traces to the service, and use a dashboard to inspect runs, run evaluations, and track metrics. It is available as a managed cloud service or as a self-hosted stack started with Docker.
The backend is a FastAPI application with a Next.js dashboard, PostgreSQL 16 for storage, Redis 7 as broker and cache, and Celery workers with a Beat scheduler for background jobs. Two access planes are separated: a management plane authenticated with a bearer token from Supabase or Firebase for users, organizations, and projects, and a data plane authenticated with an API key and project name for trace ingestion and queries. Trace uploads and evaluation requests are queued and acknowledged with 202 before a worker persists spans or runs an LLM-as-a-judge call through LiteLLM.
Integrations cover agent frameworks including LangGraph, CrewAI, the Claude Agent SDK, and the OpenAI Agents SDK. The project positions itself around self-repairing agents: trace evidence and outcome evaluations feed a repair loop that proposes candidate rules, tests them against real tasks, and keeps the validated ones for future runs.
Features
- Tracing: capture agent runs as traces and spans through the SDK or HTTP, with filtered and paginated queries over traces and sessions
- Evaluations: asynchronous LLM-as-a-judge scoring of outcomes, executed by background workers
- Monitoring dashboard: a Next.js UI for browsing runs, sessions, and metrics
- Framework integrations: LangGraph, CrewAI, Claude Agent SDK, OpenAI Agents SDK, and others
- Organizations and projects: identity service with organization and project scoping, backed by Supabase or Firebase auth
- Self-hosting: Docker-based stack of frontend, API, worker, scheduler, PostgreSQL, and Redis
- API reference: an interactive Scalar reference served alongside the API
- Validated repair rules: failures become candidate rules that are tested before reuse across workflows
