Iterative Reweighted Least Squares Networks With Convergence Guarantees
  for Solving Inverse Imaging Problems
Iaroslav Koshelev Stamatios Lefkimmiatis
In this work we present a novel optimization strategy for image
reconstruction tasks under analysis-based image regularization, which promotes
sparse and/or low-rank solutions in some learned transform domain. We
parameterize such regularizers using potential functions that correspond to
weighted extensions of the $\ell_p^p$-vector and $\mathcal{S}_p^p$
Schatten-matrix quasi-norms with $0 < p \le 1$. Our proposed minimization
strategy extends the Iteratively Reweighted Least Squares (IRLS) method,
typically used for synthesis-based $\ell_p$ and $\mathcal{S}_p$ norm and
analysis-based $\ell_1$ and nuclear norm regularization. We prove that under
mild conditions our minimization algorithm converges linearly to a stationary
point, and we provide an upper bound for its convergence rate. Further, to
select the parameters of the regularizers that deliver the best results for the
problem at hand, we propose to learn them from training data by formulating the
supervised learning process as a stochastic bilevel optimization problem. We
show that thanks to the convergence guarantees of our proposed minimization
strategy, such optimization can be successfully performed with a
memory-efficient implicit back-propagation scheme. We implement our learned
IRLS variants as recurrent networks and assess their performance on the
challenging image reconstruction tasks of non-blind deblurring,
super-resolution and demosaicking. The comparisons against other existing
learned reconstruction approaches demonstrate that our overall method is very
competitive and in many cases outperforms existing unrolled networks, whose
number of parameters is orders of magnitude higher than in our case.