LLM-Detector: Improving AI-Generated Chinese Text Detection with
  Open-Source LLM Instruction Tuning
Rongsheng Wang Haoming Chen Ruizhe Zhou Han Ma Yaofei Duan Yanlan Kang Songhua Yang Baoyu Fan Tao Tan
ChatGPT and other general large language models (LLMs) have achieved
remarkable success, but they have also raised concerns about the misuse of
AI-generated texts. Existing AI-generated text detection models, such as based
on BERT and RoBERTa, are prone to in-domain over-fitting, leading to poor
out-of-domain (OOD) detection performance. In this paper, we first collected
Chinese text responses generated by human experts and 9 types of LLMs, for
which to multiple domains questions, and further created a dataset that mixed
human-written sentences and sentences polished by LLMs. We then proposed
LLM-Detector, a novel method for both document-level and sentence-level text
detection through Instruction Tuning of LLMs. Our method leverages the wealth
of knowledge LLMs acquire during pre-training, enabling them to detect the text
they generate. Instruction tuning aligns the model's responses with the user's
expected text detection tasks. Experimental results show that previous methods
struggle with sentence-level AI-generated text detection and OOD detection. In
contrast, our proposed method not only significantly outperforms baseline
methods in both sentence-level and document-level text detection but also
demonstrates strong generalization capabilities. Furthermore, since
LLM-Detector is trained based on open-source LLMs, it is easy to customize for
deployment.