How Well Do Large Language Models Truly Ground?
Hyunji Lee Sejune Joo Chaeeun Kim Joel Jang Doyoung Kim Kyoung-Woon On Minjoon Seo
Reliance on the inherent knowledge of Large Language Models (LLMs) can cause
issues such as hallucinations, lack of control, and difficulties in integrating
variable knowledge. To mitigate this, LLMs can be probed to generate responses
by grounding on external context, often given as input (knowledge-augmented
models). Yet, previous research is often confined to a narrow view of the term
"grounding", often only focusing on whether the response contains the correct
answer or not, which does not ensure the reliability of the entire response. To
address this limitation, we introduce a strict definition of grounding: a model
is considered truly grounded when its responses (1) fully utilize necessary
knowledge from the provided context, and (2) don't exceed the knowledge within
the contexts. We introduce a new dataset and a grounding metric to assess this
new definition and perform experiments across 13 LLMs of different sizes and
training methods to provide insights into the factors that influence grounding
performance. Our findings contribute to a better understanding of how to
improve grounding capabilities and suggest an area of improvement toward more
reliable and controllable LLM applications.