UPB at IberLEF-2023 AuTexTification: Detection of Machine-Generated Text
  using Transformer Ensembles
Andrei-Alexandru Preda Dumitru-Clementin Cercel Traian Rebedea Costin-Gabriel Chiru
This paper describes the solutions submitted by the UPB team to the
AuTexTification shared task, featured as part of IberLEF-2023. Our team
participated in the first subtask, identifying text documents produced by large
language models instead of humans. The organizers provided a bilingual dataset
for this subtask, comprising English and Spanish texts covering multiple
domains, such as legal texts, social media posts, and how-to articles. We
experimented mostly with deep learning models based on Transformers, as well as
training techniques such as multi-task learning and virtual adversarial
training to obtain better results. We submitted three runs, two of which
consisted of ensemble models. Our best-performing model achieved macro
F1-scores of 66.63% on the English dataset and 67.10% on the Spanish dataset.