Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement
  Learning
Shengyi Huang Quentin Gallouédec Florian Felten Antonin Raffin Rousslan Fernand Julien Dossa Yanxiao Zhao Ryan Sullivan Viktor Makoviychuk Denys Makoviichuk Mohamad H. Danesh Cyril Roumégous Jiayi Weng Chufan Chen Md Masudur Rahman João G. M. Araújo Guorui Quan Daniel Tan Timo Klein Rujikorn Charakorn Mark Towers Yann Berthelot Kinal Mehta Dipam Chakraborty Arjun KG Valentin Charraut Chang Ye Zichen Liu Lucas N. Alegre Alexander Nikulin Xiao Hu Tianlin Liu Jongwook Choi Brent Yi
In many Reinforcement Learning (RL) papers, learning curves are useful
indicators to measure the effectiveness of RL algorithms. However, the complete
raw data of the learning curves are rarely available. As a result, it is
usually necessary to reproduce the experiments from scratch, which can be
time-consuming and error-prone. We present Open RL Benchmark, a set of fully
tracked RL experiments, including not only the usual data such as episodic
return, but also all algorithm-specific and system metrics. Open RL Benchmark
is community-driven: anyone can download, use, and contribute to the data. At
the time of writing, more than 25,000 runs have been tracked, for a cumulative
duration of more than 8 years. Open RL Benchmark covers a wide range of RL
libraries and reference implementations. Special care is taken to ensure that
each experiment is precisely reproducible by providing not only the full
parameters, but also the versions of the dependencies used to generate it. In
addition, Open RL Benchmark comes with a command-line interface (CLI) for easy
fetching and generating figures to present the results. In this document, we
include two case studies to demonstrate the usefulness of Open RL Benchmark in
practice. To the best of our knowledge, Open RL Benchmark is the first RL
benchmark of its kind, and the authors hope that it will improve and facilitate
the work of researchers in the field.