Beyond Detection: Visual Realism Assessment of Deepfakes
Luka Dragar Peter Peer Vitomir Štruc Borut Batagelj
In the era of rapid digitalization and artificial intelligence advancements,
the development of DeepFake technology has posed significant security and
privacy concerns. This paper presents an effective measure to assess the visual
realism of DeepFake videos. We utilize an ensemble of two Convolutional Neural
Network (CNN) models: Eva and ConvNext. These models have been trained on the
DeepFake Game Competition (DFGC) 2022 dataset and aim to predict Mean Opinion
Scores (MOS) from DeepFake videos based on features extracted from sequences of
frames. Our method secured the third place in the recent DFGC on Visual Realism
Assessment held in conjunction with the 2023 International Joint Conference on
Biometrics (IJCB 2023). We provide an over\-view of the models, data
preprocessing, and training procedures. We also report the performance of our
models against the competition's baseline model and discuss the implications of
our findings.