Hierarchical Contrastive Learning for Pattern-Generalizable Image
  Corruption Detection
Xin Feng Yifeng Xu Guangming Lu Wenjie Pei
Effective image restoration with large-size corruptions, such as blind image
inpainting, entails precise detection of corruption region masks which remains
extremely challenging due to diverse shapes and patterns of corruptions. In
this work, we present a novel method for automatic corruption detection, which
allows for blind corruption restoration without known corruption masks.
Specifically, we develop a hierarchical contrastive learning framework to
detect corrupted regions by capturing the intrinsic semantic distinctions
between corrupted and uncorrupted regions. In particular, our model detects the
corrupted mask in a coarse-to-fine manner by first predicting a coarse mask by
contrastive learning in low-resolution feature space and then refines the
uncertain area of the mask by high-resolution contrastive learning. A
specialized hierarchical interaction mechanism is designed to facilitate the
knowledge propagation of contrastive learning in different scales, boosting the
modeling performance substantially. The detected multi-scale corruption masks
are then leveraged to guide the corruption restoration. Detecting corrupted
regions by learning the contrastive distinctions rather than the semantic
patterns of corruptions, our model has well generalization ability across
different corruption patterns. Extensive experiments demonstrate following
merits of our model: 1) the superior performance over other methods on both
corruption detection and various image restoration tasks including blind
inpainting and watermark removal, and 2) strong generalization across different
corruption patterns such as graffiti, random noise or other image content.
Codes and trained weights are available at https://github.com/xyfJASON/HCL .