Deep Image Harmonization with Learnable Augmentation
Li Niu Junyan Cao Wenyan Cong Liqing Zhang
The goal of image harmonization is adjusting the foreground appearance in a
composite image to make the whole image harmonious. To construct paired
training images, existing datasets adopt different ways to adjust the
illumination statistics of foregrounds of real images to produce synthetic
composite images. However, different datasets have considerable domain gap and
the performances on small-scale datasets are limited by insufficient training
data. In this work, we explore learnable augmentation to enrich the
illumination diversity of small-scale datasets for better harmonization
performance. In particular, our designed SYthetic COmposite Network (SycoNet)
takes in a real image with foreground mask and a random vector to learn
suitable color transformation, which is applied to the foreground of this real
image to produce a synthetic composite image. Comprehensive experiments
demonstrate the effectiveness of our proposed learnable augmentation for image
harmonization. The code of SycoNet is released at
https://github.com/bcmi/SycoNet-Adaptive-Image-Harmonization.