Fighting Fire with Fire: Adversarial Prompting to Generate a
  Misinformation Detection Dataset
Shrey Satapara Parth Mehta Debasis Ganguly Sandip Modha
The recent success in language generation capabilities of large language
models (LLMs), such as GPT, Bard, Llama etc., can potentially lead to concerns
about their possible misuse in inducing mass agitation and communal hatred via
generating fake news and spreading misinformation. Traditional means of
developing a misinformation ground-truth dataset does not scale well because of
the extensive manual effort required to annotate the data. In this paper, we
propose an LLM-based approach of creating silver-standard ground-truth datasets
for identifying misinformation. Specifically speaking, given a trusted news
article, our proposed approach involves prompting LLMs to automatically
generate a summarised version of the original article. The prompts in our
proposed approach act as a controlling mechanism to generate specific types of
factual incorrectness in the generated summaries, e.g., incorrect quantities,
false attributions etc. To investigate the usefulness of this dataset, we
conduct a set of experiments where we train a range of supervised models for
the task of misinformation detection.