Causal Prompting: Debiasing Large Language Model Prompting based on
  Front-Door Adjustment
Congzhi Zhang Linhai Zhang Deyu Zhou Guoqiang Xu
Despite the significant achievements of existing prompting methods such as
in-context learning and chain-of-thought for large language models (LLMs), they
still face challenges of various biases. Traditional debiasing methods
primarily focus on the model training stage, including data augmentation-based
and reweight-based approaches, with the limitations of addressing the complex
biases of LLMs. To address such limitations, the causal relationship behind the
prompting methods is uncovered using a structural causal model, and a novel
causal prompting method based on front-door adjustment is proposed to
effectively mitigate the bias of LLMs. In specific, causal intervention is
implemented by designing the prompts without accessing the parameters and
logits of LLMs.The chain-of-thoughts generated by LLMs are employed as the
mediator variable and the causal effect between the input prompt and the output
answers is calculated through front-door adjustment to mitigate model biases.
Moreover, to obtain the representation of the samples precisely and estimate
the causal effect more accurately, contrastive learning is used to fine-tune
the encoder of the samples by aligning the space of the encoder with the LLM.
Experimental results show that the proposed causal prompting approach achieves
excellent performance on 3 natural language processing datasets on both
open-source and closed-source LLMs.