Smoothing ADMM for Sparse-Penalized Quantile Regression with Non-Convex
  Penalties
Reza Mirzaeifard Naveen K. D. Venkategowda Vinay Chakravarthi Gogineni Stefan Werner
This paper investigates quantile regression in the presence of non-convex and
non-smooth sparse penalties, such as the minimax concave penalty (MCP) and
smoothly clipped absolute deviation (SCAD). The non-smooth and non-convex
nature of these problems often leads to convergence difficulties for many
algorithms. While iterative techniques like coordinate descent and local linear
approximation can facilitate convergence, the process is often slow. This
sluggish pace is primarily due to the need to run these approximation
techniques until full convergence at each step, a requirement we term as a
\emph{secondary convergence iteration}. To accelerate the convergence speed, we
employ the alternating direction method of multipliers (ADMM) and introduce a
novel single-loop smoothing ADMM algorithm with an increasing penalty
parameter, named SIAD, specifically tailored for sparse-penalized quantile
regression. We first delve into the convergence properties of the proposed SIAD
algorithm and establish the necessary conditions for convergence.
Theoretically, we confirm a convergence rate of $o\big({k^{-\frac{1}{4}}}\big)$
for the sub-gradient bound of augmented Lagrangian. Subsequently, we provide
numerical results to showcase the effectiveness of the SIAD algorithm. Our
findings highlight that the SIAD method outperforms existing approaches,
providing a faster and more stable solution for sparse-penalized quantile
regression.