On Exploring the Reasoning Capability of Large Language Models with
  Knowledge Graphs
Pei-Chi Lo Yi-Hang Tsai Ee-Peng Lim San-Yih Hwang
This paper examines the capacity of LLMs to reason with knowledge graphs
using their internal knowledge graph, i.e., the knowledge graph they learned
during pre-training. Two research questions are formulated to investigate the
accuracy of LLMs in recalling information from pre-training knowledge graphs
and their ability to infer knowledge graph relations from context. To address
these questions, we employ LLMs to perform four distinct knowledge graph
reasoning tasks. Furthermore, we identify two types of hallucinations that may
occur during knowledge reasoning with LLMs: content and ontology hallucination.
Our experimental results demonstrate that LLMs can successfully tackle both
simple and complex knowledge graph reasoning tasks from their own memory, as
well as infer from input context.