FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control
akhaliq
Abstract
FastTD3, an enhanced RL algorithm with parallel simulation and distributional critic, significantly accelerates training for humanoid robots.
Reinforcement learning (RL) has driven significant progress in robotics, but its complexity and long training times remain major bottlenecks. In this report, we introduce FastTD3, a simple, fast, and capable RL algorithm that significantly speeds up training for humanoid robots in popular suites such as HumanoidBench, IsaacLab, and MuJoCo Playground. Our recipe is remarkably simple: we train an off-policy TD3 agent with several modifications -- parallel simulation, large-batch updates, a distributional critic, and carefully tuned hyperparameters. FastTD3 solves a range of HumanoidBench tasks in under 3 hours on a single A100 GPU, while remaining stable during training. We also provide a lightweight and easy-to-use implementation of FastTD3 to accelerate RL research in robotics.
Community
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Accelerating Visual-Policy Learning through Parallel Differentiable Simulation (2025)
- ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrations (2025)
- Spectral Normalization for Lipschitz-Constrained Policies on Learning Humanoid Locomotion (2025)
- PTRL: Prior Transfer Deep Reinforcement Learning for Legged Robots Locomotion (2025)
- VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning (2025)
- Reduce Computational Cost In Deep Reinforcement Learning Via Randomized Policy Learning (2025)
- Effective Reinforcement Learning for Reasoning in Language Models (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper