EAGLE: A Domain Generalization Framework for AI-generated Text Detection
Amrita Bhattacharjee Raha Moraffah Joshua Garland Huan Liu
With the advancement in capabilities of Large Language Models (LLMs), one
major step in the responsible and safe use of such LLMs is to be able to detect
text generated by these models. While supervised AI-generated text detectors
perform well on text generated by older LLMs, with the frequent release of new
LLMs, building supervised detectors for identifying text from such new models
would require new labeled training data, which is infeasible in practice. In
this work, we tackle this problem and propose a domain generalization framework
for the detection of AI-generated text from unseen target generators. Our
proposed framework, EAGLE, leverages the labeled data that is available so far
from older language models and learns features invariant across these
generators, in order to detect text generated by an unknown target generator.
EAGLE learns such domain-invariant features by combining the representational
power of self-supervised contrastive learning with domain adversarial training.
Through our experiments we demonstrate how EAGLE effectively achieves
impressive performance in detecting text generated by unseen target generators,
including recent state-of-the-art ones such as GPT-4 and Claude, reaching
detection scores of within 4.7% of a fully supervised detector.