Can Large Language Models Understand Content and Propagation for
  Misinformation Detection: An Empirical Study
Mengyang Chen Lingwei Wei Han Cao Wei Zhou Songlin Hu
Large Language Models (LLMs) have garnered significant attention for their
powerful ability in natural language understanding and reasoning. In this
paper, we present a comprehensive empirical study to explore the performance of
LLMs on misinformation detection tasks. This study stands as the pioneering
investigation into the understanding capabilities of multiple LLMs regarding
both content and propagation across social media platforms. Our empirical
studies on five misinformation detection datasets show that LLMs with diverse
prompts achieve comparable performance in text-based misinformation detection
but exhibit notably constrained capabilities in comprehending propagation
structure compared to existing models in propagation-based misinformation
detection. Besides, we further design four instruction-tuned strategies to
enhance LLMs for both content and propagation-based misinformation detection.
These strategies boost LLMs to actively learn effective features from multiple
instances or hard instances, and eliminate irrelevant propagation structures,
thereby achieving better detection performance. Extensive experiments further
demonstrate LLMs would play a better capacity in content and propagation
structure under these proposed strategies and achieve promising detection
performance. These findings highlight the potential ability of LLMs to detect
misinformation.