A Framework for Bilevel Optimization on Riemannian Manifolds
Andi Han Bamdev Mishra Pratik Jawanpuria Akiko Takeda
Bilevel optimization has seen an increasing presence in various domains of
applications. In this work, we propose a framework for solving bilevel
optimization problems where variables of both lower and upper level problems
are constrained on Riemannian manifolds. We provide several hypergradient
estimation strategies on manifolds and study their estimation error. We provide
convergence and complexity analysis for the proposed hypergradient descent
algorithm on manifolds. We also extend the developments to stochastic bilevel
optimization and to the use of general retraction. We showcase the utility of
the proposed framework on various applications.