SuperInpaint: Learning Detail-Enhanced Attentional Implicit
  Representation for Super-resolutional Image Inpainting
Canyu Zhang Qing Guo Xiaoguang Li Renjie Wan Hongkai Yu Ivor Tsang Song Wang
In this work, we introduce a challenging image restoration task, referred to
as SuperInpaint, which aims to reconstruct missing regions in low-resolution
images and generate completed images with arbitrarily higher resolutions. We
have found that this task cannot be effectively addressed by stacking
state-of-the-art super-resolution and image inpainting methods as they amplify
each other's flaws, leading to noticeable artifacts. To overcome these
limitations, we propose the detail-enhanced attentional implicit representation
(DEAR) that can achieve SuperInpaint with a single model, resulting in
high-quality completed images with arbitrary resolutions. Specifically, we use
a deep convolutional network to extract the latent embedding of an input image
and then enhance the high-frequency components of the latent embedding via an
adaptive high-pass filter. This leads to detail-enhanced semantic embedding. We
further feed the semantic embedding into an unmask-attentional module that
suppresses embeddings from ineffective masked pixels. Additionally, we extract
a pixel-wise importance map that indicates which pixels should be used for
image reconstruction. Given the coordinates of a pixel we want to reconstruct,
we first collect its neighboring pixels in the input image and extract their
detail-enhanced semantic embeddings, unmask-attentional semantic embeddings,
importance values, and spatial distances to the desired pixel. Then, we feed
all the above terms into an implicit representation and generate the color of
the specified pixel. To evaluate our method, we extend three existing datasets
for this new task and build 18 meaningful baselines using SOTA inpainting and
super-resolution methods. Extensive experimental results demonstrate that our
method outperforms all existing methods by a significant margin on four widely
used metrics.