Introducing Explicit Gaze Constraints to Face Swapping
Ethan Wilson Frederick Shic Eakta Jain
Face swapping combines one face's identity with another face's non-appearance
attributes (expression, head pose, lighting) to generate a synthetic face. This
technology is rapidly improving, but falls flat when reconstructing some
attributes, particularly gaze. Image-based loss metrics that consider the full
face do not effectively capture the perceptually important, yet spatially
small, eye regions. Improving gaze in face swaps can improve naturalness and
realism, benefiting applications in entertainment, human computer interaction,
and more. Improved gaze will also directly improve Deepfake detection efforts,
serving as ideal training data for classifiers that rely on gaze for
classification. We propose a novel loss function that leverages gaze prediction
to inform the face swap model during training and compare against existing
methods. We find all methods to significantly benefit gaze in resulting face
swaps.