BYOC: Personalized Few-Shot Classification with Co-Authored Class
  Descriptions
Arth Bohra Govert Verkes Artem Harutyunyan Pascal Weinberger Giovanni Campagna
Text classification is a well-studied and versatile building block for many
NLP applications. Yet, existing approaches require either large annotated
corpora to train a model with or, when using large language models as a base,
require carefully crafting the prompt as well as using a long context that can
fit many examples. As a result, it is not possible for end-users to build
classifiers for themselves. To address this issue, we propose a novel approach
to few-shot text classification using an LLM. Rather than few-shot examples,
the LLM is prompted with descriptions of the salient features of each class.
These descriptions are coauthored by the user and the LLM interactively: while
the user annotates each few-shot example, the LLM asks relevant questions that
the user answers. Examples, questions, and answers are summarized to form the
classification prompt. Our experiments show that our approach yields high
accuracy classifiers, within 82% of the performance of models trained with
significantly larger datasets while using only 1% of their training sets.
Additionally, in a study with 30 participants, we show that end-users are able
to build classifiers to suit their specific needs. The personalized classifiers
show an average accuracy of 90%, which is 15% higher than the state-of-the-art
approach.