Benchmarking the Generation of Fact Checking Explanations
Daniel Russo Serra Sinem Tekiroglu Marco Guerini
Fighting misinformation is a challenging, yet crucial, task. Despite the
growing number of experts being involved in manual fact-checking, this activity
is time-consuming and cannot keep up with the ever-increasing amount of Fake
News produced daily. Hence, automating this process is necessary to help curb
misinformation. Thus far, researchers have mainly focused on claim veracity
classification. In this paper, instead, we address the generation of
justifications (textual explanation of why a claim is classified as either true
or false) and benchmark it with novel datasets and advanced baselines. In
particular, we focus on summarization approaches over unstructured knowledge
(i.e. news articles) and we experiment with several extractive and abstractive
strategies. We employed two datasets with different styles and structures, in
order to assess the generalizability of our findings. Results show that in
justification production summarization benefits from the claim information,
and, in particular, that a claim-driven extractive step improves abstractive
summarization performances. Finally, we show that although cross-dataset
experiments suffer from performance degradation, a unique model trained on a
combination of the two datasets is able to retain style information in an
efficient manner.