D$^3$: Scaling Up Deepfake Detection by Learning from Discrepancy
Yongqi Yang Zhihao Qian Ye Zhu Yu Wu
The boom of Generative AI brings opportunities entangled with risks and
concerns. In this work, we seek a step toward a universal deepfake detection
system with better generalization and robustness, to accommodate the
responsible deployment of diverse image generative models. We do so by first
scaling up the existing detection task setup from the one-generator to
multiple-generators in training, during which we disclose two challenges
presented in prior methodological designs. Specifically, we reveal that the
current methods tailored for training on one specific generator either struggle
to learn comprehensive artifacts from multiple generators or tend to sacrifice
their ability to identify fake images from seen generators (i.e., In-Domain
performance) to exchange the generalization for unseen generators (i.e.,
Out-Of-Domain performance). To tackle the above challenges, we propose our
Discrepancy Deepfake Detector (D$^3$) framework, whose core idea is to learn
the universal artifacts from multiple generators by introducing a parallel
network branch that takes a distorted image as extra discrepancy signal to
supplement its original counterpart. Extensive scaled-up experiments on the
merged UFD and GenImage datasets with six detection models demonstrate the
effectiveness of our framework, achieving a 5.3% accuracy improvement in the
OOD testing compared to the current SOTA methods while maintaining the ID
performance.