Why LLMs Hallucinate, and How to Get (Evidential) Closure: Perceptual,
  Intensional, and Extensional Learning for Faithful Natural Language
  Generation
Adam Bouyamourn
We show that LLMs hallucinate because their output is not constrained to be
synonymous with claims for which they have evidence: a condition that we call
evidential closure. Information about the truth or falsity of sentences is not
statistically identified in the standard neural probabilistic language model
setup, and so cannot be conditioned on to generate new strings. We then show
how to constrain LLMs to produce output that does satisfy evidential closure. A
multimodal LLM must learn about the external world (perceptual learning); it
must learn a mapping from strings to states of the world (extensional
learning); and, to achieve fluency when generalizing beyond a body of evidence,
it must learn mappings from strings to their synonyms (intensional learning).
The output of a unimodal LLM must be synonymous with strings in a validated
evidence set. Finally, we present a heuristic procedure, Learn-Babble-Prune,
that yields faithful output from an LLM by rejecting output that is not
synonymous with claims for which the LLM has evidence.