ImplicitDeepfake: Plausible Face-Swapping through Implicit Deepfake
  Generation using NeRF and Gaussian Splatting
Georgii Stanishevskii Jakub Steczkiewicz Tomasz Szczepanik Sławomir Tadeja Jacek Tabor Przemysław Spurek
Numerous emerging deep-learning techniques have had a substantial impact on
computer graphics. Among the most promising breakthroughs are the recent rise
of Neural Radiance Fields (NeRFs) and Gaussian Splatting (GS). NeRFs encode the
object's shape and color in neural network weights using a handful of images
with known camera positions to generate novel views. In contrast, GS provides
accelerated training and inference without a decrease in rendering quality by
encoding the object's characteristics in a collection of Gaussian
distributions. These two techniques have found many use cases in spatial
computing and other domains. On the other hand, the emergence of deepfake
methods has sparked considerable controversy. Such techniques can have a form
of artificial intelligence-generated videos that closely mimic authentic
footage. Using generative models, they can modify facial features, enabling the
creation of altered identities or facial expressions that exhibit a remarkably
realistic appearance to a real person. Despite these controversies, deepfake
can offer a next-generation solution for avatar creation and gaming when of
desirable quality. To that end, we show how to combine all these emerging
technologies to obtain a more plausible outcome. Our ImplicitDeepfake1 uses the
classical deepfake algorithm to modify all training images separately and then
train NeRF and GS on modified faces. Such relatively simple strategies can
produce plausible 3D deepfake-based avatars.