Optimizing ADMM and Over-Relaxed ADMM Parameters for Linear Quadratic
  Problems
Jintao Song Wenqi Lu Yunwen Lei Yuchao Tang Zhenkuan Pan Jinming Duan
The Alternating Direction Method of Multipliers (ADMM) has gained significant
attention across a broad spectrum of machine learning applications.
Incorporating the over-relaxation technique shows potential for enhancing the
convergence rate of ADMM. However, determining optimal algorithmic parameters,
including both the associated penalty and relaxation parameters, often relies
on empirical approaches tailored to specific problem domains and contextual
scenarios. Incorrect parameter selection can significantly hinder ADMM's
convergence rate. To address this challenge, in this paper we first propose a
general approach to optimize the value of penalty parameter, followed by a
novel closed-form formula to compute the optimal relaxation parameter in the
context of linear quadratic problems (LQPs). We then experimentally validate
our parameter selection methods through random instantiations and diverse
imaging applications, encompassing diffeomorphic image registration, image
deblurring, and MRI reconstruction.