NVIDIA NeMo Agent Toolkit is a Python library that works alongside existing agent frameworks rather than replacing them. It adds instrumentation for profiling, observing, evaluating, and optimizing agents built with LangChain, LlamaIndex, CrewAI, Semantic Kernel, Agent Development Kit (ADK), AutoGen, custom frameworks, or plain Python agents. Every agent, tool, and workflow is a composable function call, so components built once can be reused across workflows.
Workflows are declared in a YAML configuration that names the functions (tools), the LLMs, and the agent type, such as a ReAct agent, and they run from a CLI. A profiler traces a workflow from the agent level down to individual tokens and timings to find bottlenecks, and an evaluation system checks workflow accuracy offline. A built-in chat UI lets you interact with agents and debug output.
Optimization goes beyond measurement: a hyper-parameter and prompt optimizer searches for better configurations, reinforcement-learning fine-tuning trains an LLM for a specific agent, and Agent Performance Primitives add parallel execution, speculative branching, and node-level priority routing to graph-based frameworks such as LangChain, CrewAI, and Agno. An experimental NVIDIA Dynamo integration applies runtime hints for cache control and load-aware routing.
For connectivity, the toolkit acts as both a Model Context Protocol (MCP) client and server (including a FastMCP runtime), and as an Agent-to-Agent (A2A) client and server with authentication. Observability integrations cover LangSmith, Phoenix, Weave, Langfuse, and OpenTelemetry-based systems, and a public plugin API lets third parties such as Tavily and Redis ship their own integrations.
Features
- Framework agnostic: instruments agents built with LangChain, LlamaIndex, CrewAI, Semantic Kernel, ADK, AutoGen, or plain Python
- Profiling: traces whole workflows down to tool, agent, and token level with timings
- Observability: integrations for LangSmith, Phoenix, Weave, Langfuse, and OpenTelemetry-based systems
- Evaluation: offline evaluation tools for workflow accuracy
- Optimizer: automatic search over hyper-parameters and prompts
- Reinforcement-learning fine-tuning: trains an LLM on a specific agent's workflow
- Agent Performance Primitives: parallel execution, speculative branching, and priority routing for graph-based frameworks
- MCP support: use remote MCP tools or publish workflows as MCP servers, including through FastMCP
- A2A support: delegate to remote A2A agents or publish workflows as discoverable A2A agents
- YAML workflows: agents, tools, and LLMs declared in configuration and run from a CLI
- Plugin API: a public interface for third-party integrations such as Tavily and Redis
- Coding agent skills: task-specific guidance for coding agents building and evaluating workflows
