Prompted Zero-Shot Multi-label Classification of Factual Incorrectness
  in Machine-Generated Summaries
Aniket Deroy Subhankar Maity Saptarshi Ghosh
This study addresses the critical issue of factual inaccuracies in
machine-generated text summaries, an increasingly prevalent issue in
information dissemination. Recognizing the potential of such errors to
compromise information reliability, we investigate the nature of factual
inconsistencies across machine-summarized content. We introduce a prompt-based
classification system that categorizes errors into four distinct types:
misrepresentation, inaccurate quantities or measurements, false attribution,
and fabrication. The participants are tasked with evaluating a corpus of
machine-generated summaries against their original articles. Our methodology
employs qualitative judgements to identify the occurrence of factual
distortions. The results show that our prompt-based approaches are able to
detect the type of errors in the summaries to some extent, although there is
scope for improvement in our classification systems.