Neural Degradation Representation Learning for All-In-One Image
  Restoration
Mingde Yao Ruikang Xu Yuanshen Guan Jie Huang Zhiwei Xiong
Existing methods have demonstrated effective performance on a single
degradation type. In practical applications, however, the degradation is often
unknown, and the mismatch between the model and the degradation will result in
a severe performance drop. In this paper, we propose an all-in-one image
restoration network that tackles multiple degradations. Due to the
heterogeneous nature of different types of degradations, it is difficult to
process multiple degradations in a single network. To this end, we propose to
learn a neural degradation representation (NDR) that captures the underlying
characteristics of various degradations. The learned NDR decomposes different
types of degradations adaptively, similar to a neural dictionary that
represents basic degradation components. Subsequently, we develop a degradation
query module and a degradation injection module to effectively recognize and
utilize the specific degradation based on NDR, enabling the all-in-one
restoration ability for multiple degradations. Moreover, we propose a
bidirectional optimization strategy to effectively drive NDR to learn the
degradation representation by optimizing the degradation and restoration
processes alternately. Comprehensive experiments on representative types of
degradations (including noise, haze, rain, and downsampling) demonstrate the
effectiveness and generalization capability of our method.