High-Fidelity Face Swapping with Style Blending
Xinyu Yang Hongbo Bo
Face swapping has gained significant traction, driven by the plethora of
human face synthesis facilitated by deep learning methods. However, previous
face swapping methods that used generative adversarial networks (GANs) as
backbones have faced challenges such as inconsistency in blending, distortions,
artifacts, and issues with training stability. To address these limitations, we
propose an innovative end-to-end framework for high-fidelity face swapping.
First, we introduce a StyleGAN-based facial attributes encoder that extracts
essential features from faces and inverts them into a latent style code,
encapsulating indispensable facial attributes for successful face swapping.
Second, we introduce an attention-based style blending module to effectively
transfer Face IDs from source to target. To ensure accurate and quality
transferring, a series of constraint measures including contrastive face ID
learning, facial landmark alignment, and dual swap consistency is implemented.
Finally, the blended style code is translated back to the image space via the
style decoder, which is of high training stability and generative capability.
Extensive experiments on the CelebA-HQ dataset highlight the superior visual
quality of generated images from our face-swapping methodology when compared to
other state-of-the-art methods, and the effectiveness of each proposed module.
Source code and weights will be publicly available.