Dual-Teacher De-biasing Distillation Framework for Multi-domain Fake
  News Detection
Jiayang Li Xuan Feng Tianlong Gu Liang Chang
Multi-domain fake news detection aims to identify whether various news from
different domains is real or fake and has become urgent and important. However,
existing methods are dedicated to improving the overall performance of fake
news detection, ignoring the fact that unbalanced data leads to disparate
treatment for different domains, i.e., the domain bias problem. To solve this
problem, we propose the Dual-Teacher De-biasing Distillation framework (DTDBD)
to mitigate bias across different domains. Following the knowledge distillation
methods, DTDBD adopts a teacher-student structure, where pre-trained large
teachers instruct a student model. In particular, the DTDBD consists of an
unbiased teacher and a clean teacher that jointly guide the student model in
mitigating domain bias and maintaining performance. For the unbiased teacher,
we introduce an adversarial de-biasing distillation loss to instruct the
student model in learning unbiased domain knowledge. For the clean teacher, we
design domain knowledge distillation loss, which effectively incentivizes the
student model to focus on representing domain features while maintaining
performance. Moreover, we present a momentum-based dynamic adjustment algorithm
to trade off the effects of two teachers. Extensive experiments on Chinese and
English datasets show that the proposed method substantially outperforms the
state-of-the-art baseline methods in terms of bias metrics while guaranteeing
competitive performance.