Distinguishing Translations by Human, NMT, and ChatGPT: A Linguistic and
  Statistical Approach
Zhaokun Jiang Qianxi Lv Ziyin Zhang
The growing popularity of neural machine translation (NMT) and LLMs
represented by ChatGPT underscores the need for a deeper understanding of their
distinct characteristics and relationships. Such understanding is crucial for
language professionals and researchers to make informed decisions and tactful
use of these cutting-edge translation technology, but remains underexplored.
This study aims to fill this gap by investigating three key questions: (1) the
distinguishability of ChatGPT-generated translations from NMT and human
translation (HT), (2) the linguistic characteristics of each translation type,
and (3) the degree of resemblance between ChatGPT-produced translations and HT
or NMT. To achieve these objectives, we employ statistical testing, machine
learning algorithms, and multidimensional analysis (MDA) to analyze
Spokesperson's Remarks and their translations. After extracting a wide range of
linguistic features, supervised classifiers demonstrate high accuracy in
distinguishing the three translation types, whereas unsupervised clustering
techniques do not yield satisfactory results. Another major finding is that
ChatGPT-produced translations exhibit greater similarity with NMT than HT in
most MDA dimensions, which is further corroborated by distance computing and
visualization. These novel insights shed light on the interrelationships among
the three translation types and have implications for the future advancements
of NMT and generative AI.