Deep Learning-based Image and Video Inpainting: A Survey
Weize Quan Jiaxi Chen Yanli Liu Dong-Ming Yan Peter Wonka
Image and video inpainting is a classic problem in computer vision and
computer graphics, aiming to fill in the plausible and realistic content in the
missing areas of images and videos. With the advance of deep learning, this
problem has achieved significant progress recently. The goal of this paper is
to comprehensively review the deep learning-based methods for image and video
inpainting. Specifically, we sort existing methods into different categories
from the perspective of their high-level inpainting pipeline, present different
deep learning architectures, including CNN, VAE, GAN, diffusion models, etc.,
and summarize techniques for module design. We review the training objectives
and the common benchmark datasets. We present evaluation metrics for low-level
pixel and high-level perceptional similarity, conduct a performance evaluation,
and discuss the strengths and weaknesses of representative inpainting methods.
We also discuss related real-world applications. Finally, we discuss open
challenges and suggest potential future research directions.