The Missing Piece in Model Editing: A Deep Dive into the Hidden Damage
  Brought By Model Editing
Jianchen Wang Zhouhong Gu Zhuozhi Xiong Hongwei Feng Yanghua Xiao
Large Language Models have revolutionized numerous tasks with their
remarkable efficacy.However, the editing of these models, crucial for
rectifying outdated or erroneous information, often leads to a complex issue
known as the ripple effect in the hidden space. This effect, while difficult to
detect, can significantly impede the efficacy of model editing tasks and
deteriorate model performance.This paper addresses this scientific challenge by
proposing a novel evaluation methodology, Graphical Outlier Relation based
Assessment(GORA), which quantitatively evaluates the adaptations of the model
and the subsequent impact of editing. Furthermore, we introduce the Selective
Outlier Re-Editing Approach(SORA), a model editing method designed to mitigate
this ripple effect. Our comprehensive evaluations reveal that the ripple effect
in the hidden space is a significant issue in all current model editing
methods. However, our proposed methods, GORA and SORA, effectively identify and
alleviate this issue, respectively, contributing to the advancement of LLM
editing techniques.