MULTITuDE: Large-Scale Multilingual Machine-Generated Text Detection
  Benchmark
Dominik Macko Robert Moro Adaku Uchendu Jason Samuel Lucas Michiharu Yamashita Matúš Pikuliak Ivan Srba Thai Le Dongwon Lee Jakub Simko Maria Bielikova
There is a lack of research into capabilities of recent LLMs to generate
convincing text in languages other than English and into performance of
detectors of machine-generated text in multilingual settings. This is also
reflected in the available benchmarks which lack authentic texts in languages
other than English and predominantly cover older generators. To fill this gap,
we introduce MULTITuDE, a novel benchmarking dataset for multilingual
machine-generated text detection comprising of 74,081 authentic and
machine-generated texts in 11 languages (ar, ca, cs, de, en, es, nl, pt, ru,
uk, and zh) generated by 8 multilingual LLMs. Using this benchmark, we compare
the performance of zero-shot (statistical and black-box) and fine-tuned
detectors. Considering the multilinguality, we evaluate 1) how these detectors
generalize to unseen languages (linguistically similar as well as dissimilar)
and unseen LLMs and 2) whether the detectors improve their performance when
trained on multiple languages.