Controlgym: Large-Scale Safety-Critical Control Environments for
  Benchmarking Reinforcement Learning Algorithms
Xiangyuan Zhang Weichao Mao Saviz Mowlavi Mouhacine Benosman Tamer Başar
We introduce controlgym, a library of thirty-six safety-critical industrial
control settings, and ten infinite-dimensional partial differential equation
(PDE)-based control problems. Integrated within the OpenAI Gym/Gymnasium (Gym)
framework, controlgym allows direct applications of standard reinforcement
learning (RL) algorithms like stable-baselines3. Our control environments
complement those in Gym with continuous, unbounded action and observation
spaces, motivated by real-world control applications. Moreover, the PDE control
environments uniquely allow the users to extend the state dimensionality of the
system to infinity while preserving the intrinsic dynamics. This feature is
crucial for evaluating the scalability of RL algorithms for control. This
project serves the learning for dynamics & control (L4DC) community, aiming to
explore key questions: the convergence of RL algorithms in learning control
policies; the stability and robustness issues of learning-based controllers;
and the scalability of RL algorithms to high- and potentially
infinite-dimensional systems. We open-source the controlgym project at
https://github.com/xiangyuan-zhang/controlgym.