A Real-World Quadrupedal Locomotion Benchmark for Offline Reinforcement
  Learning
Hongyin Zhang Shuyu Yang Donglin Wang
Online reinforcement learning (RL) methods are often data-inefficient or
unreliable, making them difficult to train on real robotic hardware, especially
quadruped robots. Learning robotic tasks from pre-collected data is a promising
direction. Meanwhile, agile and stable legged robotic locomotion remains an
open question in their general form. Offline reinforcement learning (ORL) has
the potential to make breakthroughs in this challenging field, but its current
bottleneck lies in the lack of diverse datasets for challenging realistic
tasks. To facilitate the development of ORL, we benchmarked 11 ORL algorithms
in the realistic quadrupedal locomotion dataset. Such dataset is collected by
the classic model predictive control (MPC) method, rather than the model-free
online RL method commonly used by previous benchmarks. Extensive experimental
results show that the best-performing ORL algorithms can achieve competitive
performance compared with the model-free RL, and even surpass it in some tasks.
However, there is still a gap between the learning-based methods and MPC,
especially in terms of stability and rapid adaptation. Our proposed benchmark
will serve as a development platform for testing and evaluating the performance
of ORL algorithms in real-world legged locomotion tasks.