From human experts to machines: An LLM supported approach to ontology
  and knowledge graph construction
Vamsi Krishna Kommineni Birgitta König-Ries Sheeba Samuel
The conventional process of building Ontologies and Knowledge Graphs (KGs)
heavily relies on human domain experts to define entities and relationship
types, establish hierarchies, maintain relevance to the domain, fill the ABox
(or populate with instances), and ensure data quality (including amongst others
accuracy and completeness). On the other hand, Large Language Models (LLMs)
have recently gained popularity for their ability to understand and generate
human-like natural language, offering promising ways to automate aspects of
this process. This work explores the (semi-)automatic construction of KGs
facilitated by open-source LLMs. Our pipeline involves formulating competency
questions (CQs), developing an ontology (TBox) based on these CQs, constructing
KGs using the developed ontology, and evaluating the resultant KG with minimal
to no involvement of human experts. We showcase the feasibility of our
semi-automated pipeline by creating a KG on deep learning methodologies by
exploiting scholarly publications. To evaluate the answers generated via
Retrieval-Augmented-Generation (RAG) as well as the KG concepts automatically
extracted using LLMs, we design a judge LLM, which rates the generated content
based on ground truth. Our findings suggest that employing LLMs could
potentially reduce the human effort involved in the construction of KGs,
although a human-in-the-loop approach is recommended to evaluate automatically
generated KGs.