InstructEdit: Instruction-based Knowledge Editing for Large Language
  Models
Bozhong Tian Siyuan Cheng Xiaozhuan Liang Ningyu Zhang Yi Hu Kouying Xue Yanjie Gou Xi Chen Huajun Chen
Knowledge editing for large language models can offer an efficient solution
to alter a model's behavior without negatively impacting the overall
performance. However, the current approach encounters issues with limited
generalizability across tasks, necessitating one distinct editor for each task,
which significantly hinders the broader applications. To address this, we take
the first step to analyze the multi-task generalization issue in knowledge
editing. Specifically, we develop an instruction-based editing technique,
termed InstructEdit, which facilitates the editor's adaptation to various task
performances simultaneously using simple instructions. With only one unified
editor for each LLM, we empirically demonstrate that InstructEdit can improve
the editor's control, leading to an average 14.86% increase in Reliability in
multi-task editing setting. Furthermore, experiments involving holdout unseen
task illustrate that InstructEdit consistently surpass previous strong
baselines. To further investigate the underlying mechanisms of
instruction-based knowledge editing, we analyze the principal components of the
editing gradient directions, which unveils that instructions can help control
optimization direction with stronger OOD generalization. Code and datasets will
be available in https://github.com/zjunlp/EasyEdit.