In-Context Learning Demonstration Selection via Influence Analysis
Vinay M. S. Minh-Hao Van Xintao Wu
Large Language Models (LLMs) have demonstrated their In-Context Learning
(ICL) capabilities which provides an opportunity to perform few shot learning
without any gradient update. Despite its multiple benefits, ICL generalization
performance is sensitive to the selected demonstrations. Selecting effective
demonstrations for ICL is still an open research challenge. To address this
challenge, we propose a demonstration selection method called InfICL which
analyzes influences of training samples through influence functions.
Identifying highly influential training samples can potentially aid in
uplifting the ICL generalization performance. To limit the running cost of
InfICL, we only employ the LLM to generate sample embeddings, and don't perform
any costly fine tuning. We perform empirical study on multiple real-world
datasets and show merits of our InfICL against state-of-the-art baselines.