Contrastive Multi-FaceForensics: An End-to-end Bi-grained Contrastive
  Learning Approach for Multi-face Forgery Detection
Cong Zhang Honggang Qi Yuezun Li Siwei Lyu
DeepFakes have raised serious societal concerns, leading to a great surge in
detection-based forensics methods in recent years. Face forgery recognition is
the conventional detection method that usually follows a two-phase pipeline: it
extracts the face first and then determines its authenticity by classification.
Since DeepFakes in the wild usually contain multiple faces, using face forgery
detection methods is merely practical as they have to process faces in a
sequel, i.e., only one face is processed at the same time. One straightforward
way to address this issue is to integrate face extraction and forgery detection
in an end-to-end fashion by adapting advanced object detection architectures.
However, as these object detection architectures are designed to capture the
semantic information of different object categories rather than the subtle
forgery traces among the faces, the direct adaptation is far from optimal. In
this paper, we describe a new end-to-end framework, Contrastive
Multi-FaceForensics (COMICS), to enhance multi-face forgery detection. The core
of the proposed framework is a novel bi-grained contrastive learning approach
that explores effective face forgery traces at both the coarse- and
fine-grained levels. Specifically, the coarse-grained level contrastive
learning captures the discriminative features among positive and negative
proposal pairs in multiple scales with the instruction of the proposal
generator, and the fine-grained level contrastive learning captures the
pixel-wise discrepancy between the forged and original areas of the same face
and the pixel-wise content inconsistency between different faces. Extensive
experiments on the OpenForensics dataset demonstrate our method outperforms
other counterparts by a large margin (~18.5%) and shows great potential for
integration into various architectures.