Benchmarking Offline Reinforcement Learning on Real-Robot Hardware
Nico Gürtler Sebastian Blaes Pavel Kolev Felix Widmaier Manuel Wüthrich Stefan Bauer Bernhard Schölkopf Georg Martius
Learning policies from previously recorded data is a promising direction for
real-world robotics tasks, as online learning is often infeasible. Dexterous
manipulation in particular remains an open problem in its general form. The
combination of offline reinforcement learning with large diverse datasets,
however, has the potential to lead to a breakthrough in this challenging domain
analogously to the rapid progress made in supervised learning in recent years.
To coordinate the efforts of the research community toward tackling this
problem, we propose a benchmark including: i) a large collection of data for
offline learning from a dexterous manipulation platform on two tasks, obtained
with capable RL agents trained in simulation; ii) the option to execute learned
policies on a real-world robotic system and a simulation for efficient
debugging. We evaluate prominent open-sourced offline reinforcement learning
algorithms on the datasets and provide a reproducible experimental setup for
offline reinforcement learning on real systems.