Training Implicit Networks for Image Deblurring using Jacobian-Free
  Backpropagation
Linghai Liu Shuaicheng Tong Lisa Zhao
Recent efforts in applying implicit networks to solve inverse problems in
imaging have achieved competitive or even superior results when compared to
feedforward networks. These implicit networks only require constant memory
during backpropagation, regardless of the number of layers. However, they are
not necessarily easy to train. Gradient calculations are computationally
expensive because they require backpropagating through a fixed point. In
particular, this process requires solving a large linear system whose size is
determined by the number of features in the fixed point iteration. This paper
explores a recently proposed method, Jacobian-free Backpropagation (JFB), a
backpropagation scheme that circumvents such calculation, in the context of
image deblurring problems. Our results show that JFB is comparable against
fine-tuned optimization schemes, state-of-the-art (SOTA) feedforward networks,
and existing implicit networks at a reduced computational cost.