How are Prompts Different in Terms of Sensitivity?
Sheng Lu Hendrik Schuff Iryna Gurevych
In-context learning (ICL) has become one of the most popular learning
paradigms. While there is a growing body of literature focusing on prompt
engineering, there is a lack of systematic analysis comparing the effects of
prompts across different models and tasks. To address this gap, we present a
comprehensive prompt analysis based on the sensitivity of a function. Our
analysis reveals that sensitivity is an unsupervised proxy for model
performance, as it exhibits a strong negative correlation with accuracy. We use
gradient-based saliency scores to empirically demonstrate how different prompts
affect the relevance of input tokens to the output, resulting in different
levels of sensitivity. Furthermore, we introduce sensitivity-aware decoding
which incorporates sensitivity estimation as a penalty term in the standard
greedy decoding. We show that this approach is particularly helpful when
information in the input is scarce. Our work provides a fresh perspective on
the analysis of prompts, and contributes to a better understanding of the
mechanism of ICL.