ART (Agent Reinforcement Trainer) is an open-source reinforcement-learning framework for post-training multi-step LLM agents with GRPO. It wraps the training loop so an existing Python application can learn from its own runs: your code executes the agent, assigns a reward to each finished trajectory, and the framework updates the model from that experience.
Functionality is split between a client and a server. The OpenAI-compatible client sits in your codebase and requests completions as the agent works, recording each system, user, and assistant message into a trajectory. Requests are routed to the server, which serves the model's latest LoRA in vLLM. When rollouts finish, trajectories are grouped and sent to the server, inference pauses, the server trains with GRPO starting from the latest checkpoint or an empty LoRA, saves the new adapter, loads it into vLLM, and unblocks inference for the next iteration.
The client runs on any machine with Python, while the server runs wherever a GPU is available — a local card or an ephemeral GPU environment started by the server. A managed alternative, W&B Training (Serverless RL), handles training and inference infrastructure through a ServerlessBackend; the documented figures are 40% lower cost from multiplexing on a shared inference cluster, 28% faster training, scaling past 2,000 concurrent requests, and checkpoints made available through W&B Inference. Integrations with W&B, Langfuse, and OpenPipe cover observability.
Model coverage follows vLLM and HuggingFace transformers: most compatible causal language models work, in particular those supported by Unsloth, with Gemma 3 documented as unsupported. Example notebooks train Qwen 3.6 27B to search email using RULER, Qwen 3.6 27B to play 2048, Qwen 2.5 3B to master the NWS MCP server, and smaller Qwen models on Temporal Clue, Tic Tac Toe, and Codenames, alongside supervised fine-tuning and distillation examples and a LangGraph variant of the email agent.
Features
- GRPO training loop: rollouts, reward assignment, trajectory grouping, and LoRA training run as one alternating inference-and-training cycle
- Client/server split: an OpenAI-compatible client in your application talks to a GPU-side server that owns inference and training
- Train from anywhere: the client runs on a laptop while the server uses a local GPU or an ephemeral GPU environment
- Serverless backend: W&B Training manages the training and inference infrastructure and exposes each checkpoint through W&B Inference
- RULER rewards: automatic reward generation removes the need to hand-write a reward function for every task
- MCP·RL: models are trained to use the tools exposed by an MCP server
- AutoRL: task training without a labelled dataset, using generated inputs and RULER scoring
- Supervised fine-tuning: distillation and SFT-then-RL recipes, plus fine-tuning from a dataset file
- Observability: integrations with W&B, Langfuse, and OpenPipe for run inspection
- Configurable defaults: training parameters and inference-engine settings can be overridden or left at the supplied defaults
