Editing Knowledge Representation of Language Lodel via Rephrased Prefix
  Prompts
Yuchen Cai Ding Cao Rongxi Guo Yaqin Wen Guiquan Liu Enhong Chen
Neural language models (LMs) have been extensively trained on vast corpora to
store factual knowledge about various aspects of the world described in texts.
Current technologies typically employ knowledge editing methods or specific
prompts to modify LM outputs. However, existing knowledge editing methods are
costly and inefficient, struggling to produce appropriate text. Additionally,
prompt engineering is opaque and requires significant effort to find suitable
prompts. To address these issues, we introduce a new method called PSPEM
(Prefix Soft Prompt Editing Method), that can be used for a lifetime with just
one training. It resolves the inefficiencies and generalizability issues in
knowledge editing methods and overcomes the opacity of prompt engineering by
automatically seeking optimal soft prompts. Specifically, PSPEM utilizes a
prompt encoder and an encoding converter to refine key information in prompts
and uses prompt alignment techniques to guide model generation, ensuring text
consistency and adherence to the intended structure and content, thereby
maintaining an optimal balance between efficiency and accuracy. We have
validated the effectiveness of PSPEM through knowledge editing and attribute
inserting. On the COUNTERFACT dataset, PSPEM achieved nearly 100\% editing
accuracy and demonstrated the highest level of fluency. We further analyzed the
similarities between PSPEM and original prompts and their impact on the model's
internals. The results indicate that PSPEM can serve as an alternative to
original prompts, supporting the model in effective editing.