Integrating Audio-Visual Features for Multimodal Deepfake Detection
Sneha Muppalla Shan Jia Siwei Lyu
Deepfakes are AI-generated media in which an image or video has been
digitally modified. The advancements made in deepfake technology have led to
privacy and security issues. Most deepfake detection techniques rely on the
detection of a single modality. Existing methods for audio-visual detection do
not always surpass that of the analysis based on single modalities. Therefore,
this paper proposes an audio-visual-based method for deepfake detection, which
integrates fine-grained deepfake identification with binary classification. We
categorize the samples into four types by combining labels specific to each
single modality. This method enhances the detection under intra-domain and
cross-domain testing.