A Bayesian approach for prompt optimization in pre-trained language
  models
Antonio Sabbatella Andrea Ponti Antonio Candelieri Ilaria Giordani Francesco Archetti
A prompt is a sequence of symbol or tokens, selected from a vocabulary
according to some rule, which is prepended/concatenated to a textual query. A
key problem is how to select the sequence of tokens: in this paper we formulate
it as a combinatorial optimization problem. The high dimensionality of the
token space com-pounded by the length of the prompt sequence requires a very
efficient solution. In this paper we propose a Bayesian optimization method,
executed in a continuous em-bedding of the combinatorial space. In this paper
we focus on hard prompt tuning (HPT) which directly searches for discrete
tokens to be added to the text input with-out requiring access to the large
language model (LLM) and can be used also when LLM is available only as a
black-box. This is critically important if LLMs are made available in the Model
as a Service (MaaS) manner as in GPT-4. The current manu-script is focused on
the optimization of discrete prompts for classification tasks. The discrete
prompts give rise to difficult combinatorial optimization problem which easily
become intractable given the dimension of the token space in realistic
applications. The optimization method considered in this paper is Bayesian
optimization (BO) which has become the dominant approach in black-box
optimization for its sample efficiency along with its modular structure and
versatility. In this paper we use BoTorch, a library for Bayesian optimization
research built on top of pyTorch. Albeit preliminary and obtained using a
'vanilla' version of BO, the experiments on RoB-ERTa on six benchmarks, show a
good performance across a variety of tasks and enable an analysis of the
tradeoff between size of the search space, accuracy and wall clock time.