Deep Learning Detection Method for Large Language Models-Generated
  Scientific Content
Bushra Alhijawi Rawan Jarrar Aseel AbuAlRub Arwa Bader
Large Language Models (LLMs), such as GPT-3 and BERT, reshape how textual
content is written and communicated. These models have the potential to
generate scientific content that is indistinguishable from that written by
humans. Hence, LLMs carry severe consequences for the scientific community,
which relies on the integrity and reliability of publications. This research
paper presents a novel ChatGPT-generated scientific text detection method,
AI-Catcher. AI-Catcher integrates two deep learning models, multilayer
perceptron (MLP) and convolutional neural networks (CNN). The MLP learns the
feature representations of the linguistic and statistical features. The CNN
extracts high-level representations of the sequential patterns from the textual
content. AI-Catcher is a multimodal model that fuses hidden patterns derived
from MLP and CNN. In addition, a new ChatGPT-Generated scientific text dataset
is collected to enhance AI-generated text detection tools, AIGTxt. AIGTxt
contains 3000 records collected from published academic articles across ten
domains and divided into three classes: Human-written, ChatGPT-generated, and
Mixed text. Several experiments are conducted to evaluate the performance of
AI-Catcher. The comparative results demonstrate the capability of AI-Catcher to
distinguish between human-written and ChatGPT-generated scientific text more
accurately than alternative methods. On average, AI-Catcher improved accuracy
by 37.4%.