Neural Authorship Attribution: Stylometric Analysis on Large Language
  Models
Tharindu Kumarage Huan Liu
Large language models (LLMs) such as GPT-4, PaLM, and Llama have
significantly propelled the generation of AI-crafted text. With rising concerns
about their potential misuse, there is a pressing need for AI-generated-text
forensics. Neural authorship attribution is a forensic effort, seeking to trace
AI-generated text back to its originating LLM. The LLM landscape can be divided
into two primary categories: proprietary and open-source. In this work, we
delve into these emerging categories of LLMs, focusing on the nuances of neural
authorship attribution. To enrich our understanding, we carry out an empirical
analysis of LLM writing signatures, highlighting the contrasts between
proprietary and open-source models, and scrutinizing variations within each
group. By integrating stylometric features across lexical, syntactic, and
structural aspects of language, we explore their potential to yield
interpretable results and augment pre-trained language model-based classifiers
utilized in neural authorship attribution. Our findings, based on a range of
state-of-the-art LLMs, provide empirical insights into neural authorship
attribution, paving the way for future investigations aimed at mitigating the
threats posed by AI-generated misinformation.