Detecting AI-Generated Images via CLIP
A. G. Moskowitz T. Gaona J. Peterson
As AI-generated image (AIGI) methods become more powerful and accessible, it
has become a critical task to determine if an image is real or AI-generated.
Because AIGI lack the signatures of photographs and have their own unique
patterns, new models are needed to determine if an image is AI-generated. In
this paper, we investigate the ability of the Contrastive Language-Image
Pre-training (CLIP) architecture, pre-trained on massive internet-scale data
sets, to perform this differentiation. We fine-tune CLIP on real images and
AIGI from several generative models, enabling CLIP to determine if an image is
AI-generated and, if so, determine what generation method was used to create
it. We show that the fine-tuned CLIP architecture is able to differentiate AIGI
as well or better than models whose architecture is specifically designed to
detect AIGI. Our method will significantly increase access to AIGI-detecting
tools and reduce the negative effects of AIGI on society, as our CLIP
fine-tuning procedures require no architecture changes from publicly available
model repositories and consume significantly less GPU resources than other AIGI
detection models.