Batch-less stochastic gradient descent for compressive learning of deep
  regularization for image denoising
Hui Shi Yann Traonmilin J-F Aujol
We consider the problem of denoising with the help of prior information taken
from a database of clean signals or images. Denoising with variational methods
is very efficient if a regularizer well adapted to the nature of the data is
available. Thanks to the maximum a posteriori Bayesian framework, such
regularizer can be systematically linked with the distribution of the data.
With deep neural networks (DNN), complex distributions can be recovered from a
large training database.To reduce the computational burden of this task, we
adapt the compressive learning framework to the learning of regularizers
parametrized by DNN. We propose two variants of stochastic gradient descent
(SGD) for the recovery of deep regularization parameters from a heavily
compressed database. These algorithms outperform the initially proposed method
that was limited to low-dimensional signals, each iteration using information
from the whole database. They also benefit from classical SGD convergence
guarantees. Thanks to these improvements we show that this method can be
applied for patch based image denoising.}