RLinf is reinforcement learning infrastructure for post-training foundation models and embodied agents. It covers the full pipeline for embodied AI (data collection, supervised fine-tuning, reinforcement learning in simulation or on real robots, and deployment) and agentic AI (single-agent and multi-agent RL for search, coding, and math reasoning), and it hides the distributed programming behind one configuration so that a training job scales to many GPU nodes without code changes.
Training workflows cover PPO, Async PPO, GRPO, DAPO, Reinforce++, SAC, CrossQ, RLPD, SAC-Flow, DSRL, IQL, RECAP, STEAM, RLT, and OPD, plus full-parameter, LoRA, and VLM supervised fine-tuning with DAgger and HG-DAgger. The flexible execution model lets the system explore scheduling strategies, and the hybrid execution mode for embodied RL reaches up to 2.434x the throughput of existing frameworks. Two backend combinations are offered: FSDP with HuggingFace, SGLang, or vLLM for fast adaptation to new models, and Megatron (including Megatron-Bridge) with SGLang or vLLM for large-scale training.
On the embodied side it trains vision-language-action models (OpenPI's pi-zero and pi-zero-point-five, pi-zero-FAST, OpenVLA and OpenVLA-OFT, GR00T N1.5 to N1.7, StarVLA, Evo-1, MolmoAct2, Dexbotic, and others), vision-language models (Qwen2.5-VL, Qwen3-VL, Qwen3-VL-MoE), world models (OpenSora, Wan), and world action models (FastWAM, Cosmos3, DreamZero) across simulators such as ManiSkill, LIBERO, RoboTwin, RoboVerse, BEHAVIOR, MetaWorld, IsaacLab, CALVIN, RoboCasa, Franka-Sim, and Genesis. A unified robotics interface manages hardware, tasks, and rollout wrappers separately, with support for single- and dual-arm Franka setups, several grippers and cameras, and GELLO, SpaceMouse, and PICO VR teleoperation. Isaac Lab v3.0.0 adopts RLinf as its RL training infrastructure, and RLinf is part of the PyTorch Ecosystem.
One low-level hardware abstraction runs the same training stack on NVIDIA CUDA, AMD ROCm, Huawei Ascend CANN, Moore Threads MUSA, Kunlunxin, and Biren accelerators. End-to-end recipes reproduce published RL results from provided configs and scripts, and community projects such as Qwen-VLA, RoboTwin, RISE, and NVIDIA's i4h-workflows build on it.
Features
- Embodied RL pipeline: data collection, supervised fine-tuning, simulation RL, real-world RL, and deployment for vision-language-action and world action models
- Agentic RL: single-agent recipes (SearchR1, rStar2, AgentLightning Calc-X, Online Coder, math reasoning) and multi-agent training (WideSeek-R1)
- Algorithms: PPO, Async PPO, GRPO, DAPO, Reinforce++, SAC, CrossQ, RLPD, SAC-Flow, DSRL, IQL, RECAP, STEAM, RLT, OPD, plus full-parameter, LoRA, and VLM SFT with DAgger and HG-DAgger
- Backends: FSDP with HuggingFace, SGLang, or vLLM for prototyping, and Megatron or Megatron-Bridge with SGLang or vLLM for large-scale training
- Scaling: training scales to many GPU nodes without code changes, and the hybrid execution mode reaches up to 2.434x the throughput of existing frameworks for embodied RL
- Supported models: OpenPI pi-zero family, OpenVLA and OpenVLA-OFT, GR00T N1.5 to N1.7, StarVLA, Evo-1, MolmoAct2, Qwen2.5-VL and Qwen3-VL, OpenSora, Wan, FastWAM, Cosmos3, and DreamZero
- Simulators: ManiSkill, LIBERO (including LIBERO-Pro and LIBERO-Plus), RoboTwin, RoboVerse, BEHAVIOR, MetaWorld, IsaacLab, CALVIN, RoboCasa and RoboCasa365, Franka-Sim, EmbodiChain, Genesis, and Polaris
- Real robots: a unified robotics interface for single- and dual-arm Franka, XSquare Turtle2, DOS-W1, GimArm, AgileX Piper, and SO101, with RealSense, ZED, and Lumos cameras and GELLO, SpaceMouse, and PICO VR teleoperation
- Heterogeneous hardware: NVIDIA CUDA, AMD ROCm, Huawei Ascend CANN, Moore Threads MUSA, Kunlunxin, and Biren GPUs behind one abstraction
- Reproducible recipes: configs and scripts that reproduce published RL results, including GRPO on Moonlight-16B-A3B and RL for SD3 and Wan2.2 video generation
- Ecosystem: Isaac Lab v3.0.0 uses RLinf as its RL training infrastructure, and it is a PyTorch Ecosystem project
