Conic Descent Redux for Memory-Efficient Optimization
Bingcong Li Georgios B. Giannakis
Conic programming has well-documented merits in a gamut of signal processing
and machine learning tasks. This contribution revisits a recently developed
first-order conic descent (CD) solver, and advances it in three aspects:
intuition, theory, and algorithmic implementation. It is found that CD can
afford an intuitive geometric derivation that originates from the dual problem.
This opens the door to novel algorithmic designs, with a momentum variant of
CD, momentum conic descent (MOCO) exemplified. Diving deeper into the dual
behavior CD and MOCO reveals: i) an analytically justified stopping criterion;
and, ii) the potential to design preconditioners to speed up dual convergence.
Lastly, to scale semidefinite programming (SDP) especially for low-rank
solutions, a memory efficient MOCO variant is developed and numerically
validated.