NeMo Agent Toolkit

NVIDIA NeMo Agent Toolkit is a Python library that connects existing agents to data sources and tools and adds instrumentation around them. It is framework agnostic: it works beside LangChain, LlamaIndex, CrewAI, Microsoft Semantic Kernel, Google ADK, custom enterprise frameworks, and plain Python agents, so an existing stack does not have to be replatformed. Every agent, tool, and workflow is a function call, which makes them composable and reusable across applications.

Workflows are declared in a YAML configuration file that names the functions, the LLM, and the workflow type, then run with the nat command-line tool. Installation is a single package with optional dependency groups per framework, on Python 3.11, 3.12, or 3.13, and a chat interface is included for interacting with agents and debugging workflows.

Instrumentation is the centre of the toolkit. A profiler measures whole workflows down to individual tools, agents, and tokens to locate bottlenecks; tracing integrations cover LangSmith, Phoenix, Weave, and Langfuse alongside OpenTelemetry-based systems. On top of that sit an offline evaluation system, a hyper-parameter and prompt optimiser, reinforcement-learning fine-tuning that trains workflow knowledge into a model, and framework-agnostic performance primitives that add parallel execution, speculative branching, and node-level priority routing to graph-based frameworks such as LangChain, CrewAI, and Agno. An experimental integration with NVIDIA Dynamo infers per-request latency sensitivity from agent profiles and applies runtime hints for cache control, load-aware routing, and priority-aware serving.

Both common agent protocols are supported in each direction. As a Model Context Protocol client the toolkit consumes tools from remote MCP servers, and it can publish its own tools and workflows through an MCP server runtime or a FastMCP runtime. As an Agent-to-Agent client it delegates tasks to remote agents, and as an A2A server it publishes workflows as discoverable agents with authentication. A public plugin API lets third parties ship provider-managed integrations without depending on framework-specific internal packages.

Runtime telemetry for the nat command is opt-in through a one-time consent prompt, is always off in non-interactive contexts unless an environment variable enables it, and records only the command name, outcome, duration, exit code, exception class on failure, and Python version -- never arguments, workflow or model names, file paths, or command output.

Features

• Framework agnostic: instrumentation added around LangChain, LlamaIndex, CrewAI, Semantic Kernel, Google ADK, and custom or plain Python agents
• Function-call composition: agents, tools, and workflows are reusable function calls assembled into larger applications
• Profiling: whole-workflow measurement down to tool, agent, and token level, with timings and token counts
• Observability: LangSmith, Phoenix, Weave, and Langfuse integrations plus OpenTelemetry compatibility
• Evaluation: offline tooling to validate and maintain workflow accuracy
• Optimisation: hyper-parameter and prompt optimiser, and reinforcement-learning fine-tuning for a specific workflow
• Performance primitives: parallel execution, speculative branching, and node-level priority routing for graph-based frameworks
• MCP support: consume remote MCP tools, or publish workflows through the MCP or FastMCP server runtimes
• A2A support: delegate to remote agents as a client, or publish workflows as discoverable agents as a server
• Plugin API: a public interface for third-party, provider-managed integrations
• User interface: a chat interface for interacting with agents, viewing output, and debugging workflows
• Opt-in telemetry: consent-gated, off by default in non-interactive contexts, and limited to command metadata