Inpainting-Driven Mask Optimization for Object Removal
Kodai Shimosato Norimichi Ukita
This paper proposes a mask optimization method for improving the quality of
object removal using image inpainting. While many inpainting methods are
trained with a set of random masks, a target for inpainting may be an object,
such as a person, in many realistic scenarios. This domain gap between masks in
training and inference images increases the difficulty of the inpainting task.
In our method, this domain gap is resolved by training the inpainting network
with object masks extracted by segmentation, and such object masks are also
used in the inference step. Furthermore, to optimize the object masks for
inpainting, the segmentation network is connected to the inpainting network and
end-to-end trained to improve the inpainting performance. The effect of this
end-to-end training is further enhanced by our mask expansion loss for
achieving the trade-off between large and small masks. Experimental results
demonstrate the effectiveness of our method for better object removal using
image inpainting.