DetectGPT-SC: Improving Detection of Text Generated by Large Language
  Models through Self-Consistency with Masked Predictions
Rongsheng Wang Qi Li Sihong Xie
General large language models (LLMs) such as ChatGPT have shown remarkable
success, but it has also raised concerns among people about the misuse of
AI-generated texts. Therefore, an important question is how to detect whether
the texts are generated by ChatGPT or by humans. Existing detectors are built
on the assumption that there is a distribution gap between human-generated and
AI-generated texts. These gaps are typically identified using statistical
information or classifiers. In contrast to prior research methods, we find that
large language models such as ChatGPT exhibit strong self-consistency in text
generation and continuation. Self-consistency capitalizes on the intuition that
AI-generated texts can still be reasoned with by large language models using
the same logical reasoning when portions of the texts are masked, which differs
from human-generated texts. Using this observation, we subsequently proposed a
new method for AI-generated texts detection based on self-consistency with
masked predictions to determine whether a text is generated by LLMs. This
method, which we call DetectGPT-SC. We conducted a series of experiments to
evaluate the performance of DetectGPT-SC. In these experiments, we employed
various mask scheme, zero-shot, and simple prompt for completing masked texts
and self-consistency predictions. The results indicate that DetectGPT-SC
outperforms the current state-of-the-art across different tasks.