Can Language Models Act as Knowledge Bases at Scale?
Qiyuan He Yizhong Wang Wenya Wang
Large language models (LLMs) have demonstrated remarkable proficiency in
understanding and generating responses to complex queries through large-scale
pre-training. However, the efficacy of these models in memorizing and reasoning
among large-scale structured knowledge, especially world knowledge that
explicitly covers abundant factual information remains questionable. Addressing
this gap, our research investigates whether LLMs can effectively store, recall,
and reason with knowledge on a large scale comparable to latest knowledge bases
(KBs) such as Wikidata. Specifically, we focus on three crucial aspects to
study the viability: (1) the efficiency of LLMs with different sizes in
memorizing the exact knowledge in the large-scale KB; (2) the flexibility of
recalling the memorized knowledge in response to natural language queries; (3)
the capability to infer new knowledge through reasoning. Our findings indicate
that while LLMs hold promise as large-scale KBs capable of retrieving and
responding with flexibility, enhancements in their reasoning capabilities are
necessary to fully realize their potential.