An Element-wise RSAV Algorithm for Unconstrained Optimization Problems
Shiheng Zhang Jiahao Zhang Jie Shen Guang Lin
We present a novel optimization algorithm, element-wise relaxed scalar
auxiliary variable (E-RSAV), that satisfies an unconditional energy dissipation
law and exhibits improved alignment between the modified and the original
energy. Our algorithm features rigorous proofs of linear convergence in the
convex setting. Furthermore, we present a simple accelerated algorithm that
improves the linear convergence rate to super-linear in the univariate case. We
also propose an adaptive version of E-RSAV with Steffensen step size. We
validate the robustness and fast convergence of our algorithm through ample
numerical experiments.