Automated Claim Matching with Large Language Models: Empowering
  Fact-Checkers in the Fight Against Misinformation
Eun Cheol Choi Emilio Ferrara
In today's digital era, the rapid spread of misinformation poses threats to
public well-being and societal trust. As online misinformation proliferates,
manual verification by fact checkers becomes increasingly challenging. We
introduce FACT-GPT (Fact-checking Augmentation with Claim matching
Task-oriented Generative Pre-trained Transformer), a framework designed to
automate the claim matching phase of fact-checking using Large Language Models
(LLMs). This framework identifies new social media content that either supports
or contradicts claims previously debunked by fact-checkers. Our approach
employs GPT-4 to generate a labeled dataset consisting of simulated social
media posts. This data set serves as a training ground for fine-tuning more
specialized LLMs. We evaluated FACT-GPT on an extensive dataset of social media
content related to public health. The results indicate that our fine-tuned LLMs
rival the performance of larger pre-trained LLMs in claim matching tasks,
aligning closely with human annotations. This study achieves three key
milestones: it provides an automated framework for enhanced fact-checking;
demonstrates the potential of LLMs to complement human expertise; offers public
resources, including datasets and models, to further research and applications
in the fact-checking domain.