Evolutionary Verbalizer Search for Prompt-based Few Shot Text
  Classification
Tongtao Ling Lei Chen Yutao Lai Hai-Lin Liu
Recent advances for few-shot text classification aim to wrap textual inputs
with task-specific prompts to cloze questions. By processing them with a masked
language model to predict the masked tokens and using a verbalizer that
constructs the mapping between predicted words and target labels. This approach
of using pre-trained language models is called prompt-based tuning, which could
remarkably outperform conventional fine-tuning approach in the low-data
scenario. As the core of prompt-based tuning, the verbalizer is usually
handcrafted with human efforts or suboptimally searched by gradient descent. In
this paper, we focus on automatically constructing the optimal verbalizer and
propose a novel evolutionary verbalizer search (EVS) algorithm, to improve
prompt-based tuning with the high-performance verbalizer. Specifically,
inspired by evolutionary algorithm (EA), we utilize it to automatically evolve
various verbalizers during the evolutionary procedure and select the best one
after several iterations. Extensive few-shot experiments on five text
classification datasets show the effectiveness of our method.