Generalizable Synthetic Image Detection via Language-guided Contrastive
  Learning
Haiwei Wu Jiantao Zhou Shile Zhang
The heightened realism of AI-generated images can be attributed to the rapid
development of synthetic models, including generative adversarial networks
(GANs) and diffusion models (DMs). The malevolent use of synthetic images, such
as the dissemination of fake news or the creation of fake profiles, however,
raises significant concerns regarding the authenticity of images. Though many
forensic algorithms have been developed for detecting synthetic images, their
performance, especially the generalization capability, is still far from being
adequate to cope with the increasing number of synthetic models. In this work,
we propose a simple yet very effective synthetic image detection method via a
language-guided contrastive learning and a new formulation of the detection
problem. We first augment the training images with carefully-designed textual
labels, enabling us to use a joint image-text contrastive learning for the
forensic feature extraction. In addition, we formulate the synthetic image
detection as an identification problem, which is vastly different from the
traditional classification-based approaches. It is shown that our proposed
LanguAge-guided SynThEsis Detection (LASTED) model achieves much improved
generalizability to unseen image generation models and delivers promising
performance that far exceeds state-of-the-art competitors by +22.66% accuracy
and +15.24% AUC. The code is available at https://github.com/HighwayWu/LASTED.