Counterfactual Debiasing for Generating Factually Consistent Text
  Summaries
Chenhe Dong Yuexiang Xie Yaliang Li Ying Shen
Despite substantial progress in abstractive text summarization to generate
fluent and informative texts, the factual inconsistency in the generated
summaries remains an important yet challenging problem to be solved. In this
paper, we construct causal graphs for abstractive text summarization and
identify the intrinsic causes of the factual inconsistency, i.e., the language
bias and irrelevancy bias, and further propose a debiasing framework, named
CoFactSum, to alleviate the causal effects of these biases by counterfactual
estimation. Specifically, the proposed CoFactSum provides two counterfactual
estimation strategies, i.e., Explicit Counterfactual Masking with an explicit
dynamic masking strategy, and Implicit Counterfactual Training with an implicit
discriminative cross-attention mechanism. Meanwhile, we design a Debiasing
Degree Adjustment mechanism to dynamically adapt the debiasing degree at each
decoding step. Extensive experiments on two widely-used summarization datasets
demonstrate the effectiveness of CoFactSum in enhancing the factual consistency
of generated summaries compared with several baselines.