Fill in the ____ (a Diffusion-based Image Inpainting Pipeline)
Eyoel Gebre Krishna Saxena Timothy Tran
Image inpainting is the process of taking an image and generating lost or
intentionally occluded portions. Inpainting has countless applications
including restoring previously damaged pictures, restoring the quality of
images that have been degraded due to compression, and removing unwanted
objects/text. Modern inpainting techniques have shown remarkable ability in
generating sensible completions for images with mask occlusions. In our paper,
an overview of the progress of inpainting techniques will be provided, along
with identifying current leading approaches, focusing on their strengths and
weaknesses. A critical gap in these existing models will be addressed, focusing
on the ability to prompt and control what exactly is generated. We will
additionally justify why we think this is the natural next progressive step
that inpainting models must take, and provide multiple approaches to
implementing this functionality. Finally, we will evaluate the results of our
approaches by qualitatively checking whether they generate high-quality images
that correctly inpaint regions with the objects that they are instructed to
produce.