Raising the Bar of AI-generated Image Detection with CLIP
Davide Cozzolino Giovanni Poggi Riccardo Corvi Matthias Nießner Luisa Verdoliva
Aim of this work is to explore the potential of pre-trained vision-language
models (VLMs) for universal detection of AI-generated images. We develop a
lightweight detection strategy based on CLIP features and study its performance
in a wide variety of challenging scenarios. We find that, unlike previous
belief, it is neither necessary nor convenient to use a large domain-specific
dataset for training. On the contrary, by using only a handful of example
images from a single generative model, a CLIP-based detector exhibits a
surprising generalization ability and high robustness across several different
architectures, including recent commercial tools such as Dalle-3, Midjourney
v5, and Firefly. We match the SoTA on in-distribution data, and improve largely
above it in terms of generalization to out-of-distribution data (+6% in terms
of AUC) and robustness to impaired/laundered data (+13%). Our project is
available at https://grip-unina.github.io/ClipBased-SyntheticImageDetection/