Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
Qingru Zhang Chandan Singh Liyuan Liu Xiaodong Liu Bin Yu Jianfeng Gao Tuo Zhao
In human-written articles, we often leverage the subtleties of text style,
such as bold and italics, to guide the attention of readers. These textual
emphases are vital for the readers to grasp the conveyed information. When
interacting with large language models (LLMs), we have a similar need -
steering the model to pay closer attention to user-specified information, e.g.,
an instruction. Existing methods, however, are constrained to process plain
text and do not support such a mechanism. This motivates us to introduce PASTA
- Post-hoc Attention STeering Approach, a method that allows LLMs to read text
with user-specified emphasis marks. To this end, PASTA identifies a small
subset of attention heads and applies precise attention reweighting on them,
directing the model attention to user-specified parts. Like prompting, PASTA is
applied at inference time and does not require changing any model parameters.
Experiments demonstrate that PASTA can substantially enhance an LLM's ability
to follow user instructions or integrate new knowledge from user inputs,
leading to a significant performance improvement on a variety of tasks, e.g.,
an average accuracy improvement of 22% for LLAMA-7B. Our code is publicly
available at https://github.com/QingruZhang/PASTA .