Causal Intervention for Abstractive Related Work Generation
Jiachang Liu Qi Zhang Chongyang Shi Usman Naseem Shoujin Wang Ivor Tsang
Abstractive related work generation has attracted increasing attention in
generating coherent related work that better helps readers grasp the background
in the current research. However, most existing abstractive models ignore the
inherent causality of related work generation, leading to low quality of
generated related work and spurious correlations that affect the models'
generalizability. In this study, we argue that causal intervention can address
these limitations and improve the quality and coherence of the generated
related works. To this end, we propose a novel Causal Intervention Module for
Related Work Generation (CaM) to effectively capture causalities in the
generation process and improve the quality and coherence of the generated
related works. Specifically, we first model the relations among sentence order,
document relation, and transitional content in related work generation using a
causal graph. Then, to implement the causal intervention and mitigate the
negative impact of spurious correlations, we use do-calculus to derive ordinary
conditional probabilities and identify causal effects through CaM. Finally, we
subtly fuse CaM with Transformer to obtain an end-to-end generation model.
Extensive experiments on two real-world datasets show that causal interventions
in CaM can effectively promote the model to learn causal relations and produce
related work of higher quality and coherence.