ART (Agent Reinforcement Trainer)

ART (Agent Reinforcement Trainer) is an open-source reinforcement learning (RL) framework that trains multi-step LLM agents to learn from experience with GRPO (Group Relative Policy Optimization). It wraps RL training around existing Python applications, so an agent's own workflow becomes the training environment.

ART is split into a client and a server. The OpenAI-compatible client plugs into your code: it runs rollouts of the agentic workflow, records every system, user, and assistant message in a Trajectory, and lets your code assign each Trajectory a reward. The server runs on any machine with a GPU, serves the model's latest LoRA in vLLM, trains on grouped Trajectories with GRPO, and loads the new LoRA back into vLLM. Inference pauses during each training step, and the loop repeats for a set number of iterations.

The client can run on a laptop while the server starts an ephemeral GPU environment, or both can run on a local GPU. W&B Training (Serverless RL) is a managed alternative that runs the training and inference infrastructure, with every checkpoint available through W&B Inference. ART works with most causal language models compatible with vLLM and Hugging Face Transformers, in practice those Unsloth supports (Gemma 3 is not supported).

Related techniques include RULER, which generates rewards automatically, AutoRL, which trains on a task without labeled data, MCP•RL, which teaches a model to use the tools of an MCP server, and supervised fine-tuning (SFT) for distillation or as a warmup before RL. Example notebooks cover an email search agent (ART•E), 2048, Tic Tac Toe, Codenames, Temporal Clue, and text-to-SQL distillation, mostly on Qwen models.