Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical
  Systems
Wesley A. Suttle Vipul K. Sharma Krishna C. Kosaraju S. Sivaranjani Ji Liu Vijay Gupta Brian M. Sadler
We develop provably safe and convergent reinforcement learning (RL)
algorithms for control of nonlinear dynamical systems, bridging the gap between
the hard safety guarantees of control theory and the convergence guarantees of
RL theory. Recent advances at the intersection of control and RL follow a
two-stage, safety filter approach to enforcing hard safety constraints:
model-free RL is used to learn a potentially unsafe controller, whose actions
are projected onto safe sets prescribed, for example, by a control barrier
function. Though safe, such approaches lose any convergence guarantees enjoyed
by the underlying RL methods. In this paper, we develop a single-stage,
sampling-based approach to hard constraint satisfaction that learns RL
controllers enjoying classical convergence guarantees while satisfying hard
safety constraints throughout training and deployment. We validate the efficacy
of our approach in simulation, including safe control of a quadcopter in a
challenging obstacle avoidance problem, and demonstrate that it outperforms
existing benchmarks.