Ta'keed: The First Generative Fact-Checking System for Arabic Claims
Saud Althabiti Mohammad Ammar Alsalka Eric Atwell
This paper introduces Ta'keed, an explainable Arabic automatic fact-checking
system. While existing research often focuses on classifying claims as "True"
or "False," there is a limited exploration of generating explanations for claim
credibility, particularly in Arabic. Ta'keed addresses this gap by assessing
claim truthfulness based on retrieved snippets, utilizing two main components:
information retrieval and LLM-based claim verification. We compiled the
ArFactEx, a testing gold-labelled dataset with manually justified references,
to evaluate the system. The initial model achieved a promising F1 score of 0.72
in the classification task. Meanwhile, the system's generated explanations are
compared with gold-standard explanations syntactically and semantically. The
study recommends evaluating using semantic similarities, resulting in an
average cosine similarity score of 0.76. Additionally, we explored the impact
of varying snippet quantities on claim classification accuracy, revealing a
potential correlation, with the model using the top seven hits outperforming
others with an F1 score of 0.77.