Contrastive Pseudo Learning for Open-World DeepFake Attribution
Zhimin Sun Shen Chen Taiping Yao Bangjie Yin Ran Yi Shouhong Ding Lizhuang Ma
The challenge in sourcing attribution for forgery faces has gained widespread
attention due to the rapid development of generative techniques. While many
recent works have taken essential steps on GAN-generated faces, more
threatening attacks related to identity swapping or expression transferring are
still overlooked. And the forgery traces hidden in unknown attacks from the
open-world unlabeled faces still remain under-explored. To push the related
frontier research, we introduce a new benchmark called Open-World DeepFake
Attribution (OW-DFA), which aims to evaluate attribution performance against
various types of fake faces under open-world scenarios. Meanwhile, we propose a
novel framework named Contrastive Pseudo Learning (CPL) for the OW-DFA task
through 1) introducing a Global-Local Voting module to guide the feature
alignment of forged faces with different manipulated regions, 2) designing a
Confidence-based Soft Pseudo-label strategy to mitigate the pseudo-noise caused
by similar methods in unlabeled set. In addition, we extend the CPL framework
with a multi-stage paradigm that leverages pre-train technique and iterative
learning to further enhance traceability performance. Extensive experiments
verify the superiority of our proposed method on the OW-DFA and also
demonstrate the interpretability of deepfake attribution task and its impact on
improving the security of deepfake detection area.