Mechanisms of non-factual hallucinations in language models
Lei Yu Meng Cao Jackie Chi Kit Cheung Yue Dong
State-of-the-art language models (LMs) sometimes generate non-factual
hallucinations that misalign with world knowledge. Despite extensive efforts to
detect and mitigate hallucinations, understanding their internal mechanisms
remains elusive. Our study investigates the mechanistic causes of
hallucination, specifically non-factual ones where the LM incorrectly predicts
object attributes in response to subject-relation queries. With causal
mediation analysis and embedding space projection, we identify two general
mechanistic causes of hallucinations shared across LMs of various scales and
designs: 1) insufficient subject attribute knowledge in lower layer MLPs, and
2) failing to select the correct object attribute in upper layer attention
heads and MLPs. These two mechanisms exhibit varying degrees of subject-object
association, predictive uncertainty and perturbation robustness. Additionally,
we scrutinize LM pre-training checkpoints, revealing distinct learning dynamics
for the two mechanistic causes of hallucinations. We also highlight how
attribution features from our causal analysis can effectively construct
hallucination detectors. Our work proposes a mechanistic understanding of LM
factual errors.