RMDM: A Multilabel Fakenews Dataset for Vietnamese Evidence Verification
Hai-Long Nguyen Thi-Kieu-Trang Pham Thai-Son Le Tan-Minh Nguyen Thi-Hai-Yen Vuong Ha-Thanh Nguyen
In this study, we present a novel and challenging multilabel Vietnamese
dataset (RMDM) designed to assess the performance of large language models
(LLMs), in verifying electronic information related to legal contexts, focusing
on fake news as potential input for electronic evidence. The RMDM dataset
comprises four labels: real, mis, dis, and mal, representing real information,
misinformation, disinformation, and mal-information, respectively. By including
these diverse labels, RMDM captures the complexities of differing fake news
categories and offers insights into the abilities of different language models
to handle various types of information that could be part of electronic
evidence. The dataset consists of a total of 1,556 samples, with 389 samples
for each label. Preliminary tests on the dataset using GPT-based and BERT-based
models reveal variations in the models' performance across different labels,
indicating that the dataset effectively challenges the ability of various
language models to verify the authenticity of such information. Our findings
suggest that verifying electronic information related to legal contexts,
including fake news, remains a difficult problem for language models,
warranting further attention from the research community to advance toward more
reliable AI models for potential legal applications.