Towards Consistent Natural-Language Explanations via
  Explanation-Consistency Finetuning
Yanda Chen Chandan Singh Xiaodong Liu Simiao Zuo Bin Yu He He Jianfeng Gao
Large language models (LLMs) often generate convincing, fluent explanations.
However, different from humans, they often generate inconsistent explanations
on different inputs. For example, an LLM may generate the explanation "all
birds can fly" when answering the question "Can sparrows fly?" but meanwhile
answer "no" to the related question "Can penguins fly?". Explanations should be
consistent across related examples so that they allow a human to simulate the
LLM's decision process on multiple examples. We propose explanation-consistency
finetuning (EC-finetuning), a method that adapts LLMs to generate more
consistent natural-language explanations on related examples. EC-finetuning
involves finetuning LLMs on synthetic data that is carefully constructed to
contain consistent explanations. Across a variety of question-answering
datasets in various domains, EC-finetuning yields a 10.0% relative explanation
consistency improvement on four finetuning datasets, and generalizes to seven
out-of-distribution datasets not seen during finetuning (+4.5% relative). Code
is available at https://github.com/yandachen/explanation-consistency-finetuning .