Interpretable Multimodal Misinformation Detection with Logic Reasoning
Hui Liu Wenya Wang Haoliang Li
Multimodal misinformation on online social platforms is becoming a critical
concern due to increasing credibility and easier dissemination brought by
multimedia content, compared to traditional text-only information. While
existing multimodal detection approaches have achieved high performance, the
lack of interpretability hinders these systems' reliability and practical
deployment. Inspired by NeuralSymbolic AI which combines the learning ability
of neural networks with the explainability of symbolic learning, we propose a
novel logic-based neural model for multimodal misinformation detection which
integrates interpretable logic clauses to express the reasoning process of the
target task. To make learning effective, we parameterize symbolic logical
elements using neural representations, which facilitate the automatic
generation and evaluation of meaningful logic clauses. Additionally, to make
our framework generalizable across diverse misinformation sources, we introduce
five meta-predicates that can be instantiated with different correlations.
Results on three public datasets (Twitter, Weibo, and Sarcasm) demonstrate the
feasibility and versatility of our model.