Regret Analysis of Policy Optimization over Submanifolds for Linearly
  Constrained Online LQG
Ting-Jui Chang Shahin Shahrampour
Recent advancement in online optimization and control has provided novel
tools to study online linear quadratic regulator (LQR) problems, where cost
matrices are varying adversarially over time. However, the controller
parameterization of existing works may not satisfy practical conditions like
sparsity due to physical connections. In this work, we study online linear
quadratic Gaussian problems with a given linear constraint imposed on the
controller. Inspired by the recent work of [1] which proposed, for a linearly
constrained policy optimization of an offline LQR, a second order method
equipped with a Riemannian metric that emerges naturally in the context of
optimal control problems, we propose online optimistic Newton on manifold
(OONM) which provides an online controller based on the prediction on the first
and second order information of the function sequence. To quantify the proposed
algorithm, we leverage the notion of regret defined as the sub-optimality of
its cumulative cost to that of a (locally) minimizing controller sequence and
provide the regret bound in terms of the path-length of the minimizer sequence.
Simulation results are also provided to verify the property of OONM.