Self-Supervised Graph Transformer for Deepfake Detection
Aminollah Khormali Jiann-Shiun Yuan
Deepfake detection methods have shown promising results in recognizing
forgeries within a given dataset, where training and testing take place on the
in-distribution dataset. However, their performance deteriorates significantly
when presented with unseen samples. As a result, a reliable deepfake detection
system must remain impartial to forgery types, appearance, and quality for
guaranteed generalizable detection performance. Despite various attempts to
enhance cross-dataset generalization, the problem remains challenging,
particularly when testing against common post-processing perturbations, such as
video compression or blur. Hence, this study introduces a deepfake detection
framework, leveraging a self-supervised pre-training model that delivers
exceptional generalization ability, withstanding common corruptions and
enabling feature explainability. The framework comprises three key components:
a feature extractor based on vision Transformer architecture that is
pre-trained via self-supervised contrastive learning methodology, a graph
convolution network coupled with a Transformer discriminator, and a graph
Transformer relevancy map that provides a better understanding of manipulated
regions and further explains the model's decision. To assess the effectiveness
of the proposed framework, several challenging experiments are conducted,
including in-data distribution performance, cross-dataset, cross-manipulation
generalization, and robustness against common post-production perturbations.
The results achieved demonstrate the remarkable effectiveness of the proposed
deepfake detection framework, surpassing the current state-of-the-art
approaches.