Can We Catch the Elephant? The Evolvement of Hallucination Evaluation on
  Natural Language Generation: A Survey
Siya Qi Yulan He Zheng Yuan
Hallucination in Natural Language Generation (NLG) is like the elephant in
the room, obvious but often overlooked until recent achievements significantly
improved the fluency and grammatical accuracy of generated text. For Large
Language Models (LLMs), hallucinations can happen in various downstream tasks
and casual conversations, which need accurate assessment to enhance reliability
and safety. However, current studies on hallucination evaluation vary greatly,
and people still find it difficult to sort out and select the most appropriate
evaluation methods. Moreover, as NLP research gradually shifts to the domain of
LLMs, it brings new challenges to this direction. This paper provides a
comprehensive survey on the evolvement of hallucination evaluation methods,
aiming to address three key aspects: 1) Diverse definitions and granularity of
facts; 2) The categories of automatic evaluators and their applicability; 3)
Unresolved issues and future directions.