Leveraging Large Language Models for Concept Graph Recovery and Question
  Answering in NLP Education
Rui Yang Boming Yang Sixun Ouyang Tianwei She Aosong Feng Yuang Jiang Freddy Lecue Jinghui Lu Irene Li
In the domain of Natural Language Processing (NLP), Large Language Models
(LLMs) have demonstrated promise in text-generation tasks. However, their
educational applications, particularly for domain-specific queries, remain
underexplored. This study investigates LLMs' capabilities in educational
scenarios, focusing on concept graph recovery and question-answering (QA). We
assess LLMs' zero-shot performance in creating domain-specific concept graphs
and introduce TutorQA, a new expert-verified NLP-focused benchmark for
scientific graph reasoning and QA. TutorQA consists of five tasks with 500 QA
pairs. To tackle TutorQA queries, we present CGLLM, a pipeline integrating
concept graphs with LLMs for answering diverse questions. Our results indicate
that LLMs' zero-shot concept graph recovery is competitive with supervised
methods, showing an average 3% F1 score improvement. In TutorQA tasks, LLMs
achieve up to 26% F1 score enhancement. Moreover, human evaluation and analysis
show that CGLLM generates answers with more fine-grained concepts.