Optimistic Online Non-stochastic Control via FTRL
Naram Mhaisen George Iosifidis
This paper brings the concept of "optimism" to the new and promising
framework of online Non-stochastic Control (NSC). Namely, we study how can NSC
benefit from a prediction oracle of unknown quality responsible for forecasting
future costs. The posed problem is first reduced to an optimistic learning with
delayed feedback problem, which is handled through the Optimistic Follow the
Regularized Leader (OFTRL) algorithmic family. This reduction enables the
design of OptFTRL-C, the first Disturbance Action Controller (DAC) with
optimistic policy regret bounds. These new bounds are commensurate with the
oracle's accuracy, ranging from $\mathcal{O}(1)$ for perfect predictions to the
order-optimal $\mathcal{O}(\sqrt{T})$ even when all predictions fail. By
addressing the challenge of incorporating untrusted predictions into control
systems, our work contributes to the advancement of the NSC framework and paves
the way towards effective and robust learning-based controllers.