RLinf is infrastructure for reinforcement-learning post-training of foundation models, aimed at embodied and agentic AI. It hides the distributed-programming detail so a training workflow scales to many GPU nodes without code changes, and covers PPO, asynchronous PPO, GRPO, DAPO, Reinforce++, SAC, and further algorithms alongside supervised fine-tuning.
Training and rollout engines are paired as swappable backends: FSDP with Hugging Face, SGLang, or vLLM for fast prototyping, and Megatron with SGLang or vLLM for large-scale runs. Placement can be collocated, disaggregated, or hybrid, with automatic online scaling; the hybrid execution mode for embodied RL is reported at up to 2.434× the throughput of comparable frameworks. Accelerator support extends to Moore Threads MUSA, Huawei Ascend CANN, and AMD ROCm.
On the embodied side it runs supervised fine-tuning, simulation RL, and real-world RL for vision-language-action and world-action models. Simulators include ManiSkill, LIBERO and its Pro and Plus variants, RoboTwin, RoboVerse, BEHAVIOR, MetaWorld, IsaacLab, CALVIN, RoboCasa, Franka-Sim, EmbodiChain, Genesis, and Polaris. Supported policies include π₀ and π₀.₅, OpenVLA and OpenVLA-OFT, GR00T N1.5 to N1.7, Dexbotic, StarVLA, LingBot-VLA, Evo-1, MolmoAct2, Qwen2.5-VL and Qwen3-VL, the OpenSora and Wan world models, and the Cosmos3 and DreamZero world action models.
Real-world training covers the pipeline from data collection through fine-tuning and RL to deployment, on single- and dual-arm Franka setups with Intel RealSense or Stereolabs ZED cameras and Franka, Ruiyan, or Robotiq grippers, plus XSquare Turtle2, DOS-W1, and GimArm platforms; teleoperated data collection uses GELLO, SpaceMouse, or PICO VR.
The agentic side ships single-agent recipes — SearchR1, rStar2, AgentLightning Calc-X, online coding, and math reasoning — and multi-agent training through WideSeek-R1. The project is part of the PyTorch ecosystem, is adopted by Isaac Lab v3.0.0 as its reinforcement-learning training infrastructure, and installs either as a library from PyPI or from a provided Docker image.
Features
- RL algorithms: GRPO, PPO, async PPO, DAPO, Reinforce++, SAC, CrossQ, RLPD, SAC-Flow, DSRL, IQL, RECAP, STEAM, RLT, and OPD
- Fine-tuning modes: full-parameter and LoRA supervised fine-tuning, vision-language-model fine-tuning, DAgger, and HG-DAgger
- Swappable backends: FSDP with Hugging Face, SGLang, or vLLM for prototyping; Megatron with SGLang or vLLM at scale
- Placement modes: collocated, disaggregated, and hybrid placement across GPUs and nodes, with automatic online scaling
- Simulator coverage: ManiSkill, LIBERO, RoboTwin, RoboVerse, BEHAVIOR, MetaWorld, IsaacLab, CALVIN, RoboCasa, Genesis, Polaris, and more
- Policy coverage: vision-language-action, vision-language, world, and world action models, plus custom MLP and CNN policies
- Real-world robotics: Franka arms with several camera and gripper combinations, XSquare Turtle2, DOS-W1, and GimArm
- Data collection: GELLO, SpaceMouse, and PICO VR teleoperation
- Agentic recipes: search, reasoning, coding, and multi-agent training configurations
- Multi-vendor accelerators: Moore Threads MUSA, Huawei Ascend CANN, and AMD ROCm
- Reproducible recipes: end-to-end configs and scripts that reproduce the reported state-of-the-art results
- Ecosystem integration: part of the PyTorch ecosystem and used as Isaac Lab's RL training infrastructure
