Actor-Critic Physics-informed Neural Lyapunov Control
Jiarui Wang Mahyar Fazlyab
Designing control policies for stabilization tasks with provable guarantees
is a long-standing problem in nonlinear control. A crucial performance metric
is the size of the resulting region of attraction, which essentially serves as
a robustness "margin" of the closed-loop system against uncertainties. In this
paper, we propose a new method to train a stabilizing neural network controller
along with its corresponding Lyapunov certificate, aiming to maximize the
resulting region of attraction while respecting the actuation constraints.
Crucial to our approach is the use of Zubov's Partial Differential Equation
(PDE), which precisely characterizes the true region of attraction of a given
control policy. Our framework follows an actor-critic pattern where we
alternate between improving the control policy (actor) and learning a Zubov
function (critic). Finally, we compute the largest certifiable region of
attraction by invoking an SMT solver after the training procedure. Our
numerical experiments on several design problems show consistent and
significant improvements in the size of the resulting region of attraction.