ReliableSwap: Boosting General Face Swapping Via Reliable Supervision
Ge Yuan Maomao Li Yong Zhang Huicheng Zheng
Almost all advanced face swapping approaches use reconstruction as the proxy
task, i.e., supervision only exists when the target and source belong to the
same person. Otherwise, lacking pixel-level supervision, these methods struggle
for source identity preservation. This paper proposes to construct reliable
supervision, dubbed cycle triplets, which serves as the image-level guidance
when the source identity differs from the target one during training.
Specifically, we use face reenactment and blending techniques to synthesize the
swapped face from real images in advance, where the synthetic face preserves
source identity and target attributes. However, there may be some artifacts in
such a synthetic face. To avoid the potential artifacts and drive the
distribution of the network output close to the natural one, we reversely take
synthetic images as input while the real face as reliable supervision during
the training stage of face swapping. Besides, we empirically find that the
existing methods tend to lose lower-face details like face shape and mouth from
the source. This paper additionally designs a FixerNet, providing
discriminative embeddings of lower faces as an enhancement. Our face swapping
framework, named ReliableSwap, can boost the performance of any existing face
swapping network with negligible overhead. Extensive experiments demonstrate
the efficacy of our ReliableSwap, especially in identity preservation. The
project page is https://reliable-swap.github.io/.