AgentOps

AgentOps is an observability and developer-tool platform for AI agents that covers building, evaluating, and monitoring from prototype to production. A Python SDK records sessions, LLM calls, tool use, and multi-agent interactions, and a dashboard shows step-by-step execution graphs, session replays, chat viewers, and summary analytics. The application (dashboard and API backend) is open source and can be self-hosted.

Instrumentation starts with agentops.init(apikey) at program start and agentops.endsession() at the end, which captures analytics on all LLM calls. For finer structure, decorators from agentops.sdk.decorators mark a @session root span, @agent classes, @operation or @task functions, and @workflow functions, and they nest to build a span hierarchy. All decorators record inputs and outputs, handle exceptions, and support async, generator, and custom-attributed functions.

Framework integrations are switched on by setting AGENTOPSAPIKEY in the environment and initialising the client. CrewAI installs with the crewai[agentops] extra, AG2 (formerly AutoGen) and Camel AI pick up the key automatically, LangChain uses a LangchainCallbackHandler, and the OpenAI Agents SDK is supported in both Python and TypeScript.

Features

• Replay analytics and debugging: step-by-step agent execution graphs and session replays with metadata drill-down
• LLM cost management: spend tracked per foundation-model provider
• Decorator-based spans: @session, @agent, @operation/@task, and @workflow with nesting for hierarchy
• Framework integrations: OpenAI Agents SDK (Python and TypeScript), CrewAI, AG2, Agno, LangGraph, LangChain, Camel AI, LlamaIndex, and Microsoft
• Provider integrations: OpenAI, Anthropic (Python SDK 0.32.0+), Cohere (5.4.0+), Mistral, Groq, and Ollama, including streaming and async clients
• Self-hosting: run the dashboard and API backend on your own infrastructure
• MCP server: an AgentOps MCP server published on Smithery