Knowledge Graph Enhanced Large Language Model Editing
Mengqi Zhang Xiaotian Ye Qiang Liu Pengjie Ren Shu Wu Zhumin Chen
Large language models (LLMs) are pivotal in advancing natural language
processing (NLP) tasks, yet their efficacy is hampered by inaccuracies and
outdated knowledge. Model editing emerges as a promising solution to address
these challenges. However, existing editing methods struggle to track and
incorporate changes in knowledge associated with edits, which limits the
generalization ability of postedit LLMs in processing edited knowledge. To
tackle these problems, we propose a novel model editing method that leverages
knowledge graphs for enhancing LLM editing, namely GLAME. Specifically, we
first utilize a knowledge graph augmentation module to uncover associated
knowledge that has changed due to editing, obtaining its internal
representations within LLMs. This approach allows knowledge alterations within
LLMs to be reflected through an external graph structure. Subsequently, we
design a graph-based knowledge edit module to integrate structured knowledge
into the model editing. This ensures that the updated parameters reflect not
only the modifications of the edited knowledge but also the changes in other
associated knowledge resulting from the editing process. Comprehensive
experiments conducted on GPT-J and GPT-2 XL demonstrate that GLAME
significantly improves the generalization capabilities of post-edit LLMs in
employing edited knowledge.