Emptying the Ocean with a Spoon: Should We Edit Models?
Yuval Pinter Michael Elhadad
We call into question the recently popularized method of direct model editing
as a means of correcting factual errors in LLM generations. We contrast model
editing with three similar but distinct approaches that pursue better defined
objectives: (1) retrieval-based architectures, which decouple factual memory
from inference and linguistic capabilities embodied in LLMs; (2) concept
erasure methods, which aim at preventing systemic bias in generated text; and
(3) attribution methods, which aim at grounding generations into identified
textual sources. We argue that direct model editing cannot be trusted as a
systematic remedy for the disadvantages inherent to LLMs, and while it has
proven potential in improving model explainability, it opens risks by
reinforcing the notion that models can be trusted for factuality. We call for
cautious promotion and application of model editing as part of the LLM
deployment process, and for responsibly limiting the use cases of LLMs to those
not relying on editing as a critical component.