Robust Stochastically-Descending Unrolled Networks
Samar Hadou Navid NaderiAlizadeh Alejandro Ribeiro
Deep unrolling, or unfolding, is an emerging learning-to-optimize method that
unrolls a truncated iterative algorithm in the layers of a trainable neural
network. However, the convergence guarantees and generalizability of the
unrolled networks are still open theoretical problems. To tackle these
problems, we provide deep unrolled architectures with a stochastic descent
nature by imposing descending constraints during training. The descending
constraints are forced layer by layer to ensure that each unrolled layer takes,
on average, a descent step toward the optimum during training. We theoretically
prove that the sequence constructed by the outputs of the unrolled layers is
then guaranteed to converge for unseen problems, assuming no distribution shift
between training and test problems. We also show that standard unrolling is
brittle to perturbations, and our imposed constraints provide the unrolled
networks with robustness to additive noise and perturbations. We numerically
assess unrolled architectures trained under the proposed constraints in two
different applications, including the sparse coding using learnable iterative
shrinkage and thresholding algorithm (LISTA) and image inpainting using
proximal generative flow (GLOW-Prox), and demonstrate the performance and
robustness benefits of the proposed method.