DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Framework
Keonwoo Kim Younggun Lee
With the growing volume of diverse information, the demand for classifying
arbitrary topics has become increasingly critical. To address this challenge,
we introduce DRAFT, a simple framework designed to train a classifier for
few-shot topic classification. DRAFT uses a few examples of a specific topic as
queries to construct Customized dataset with a dense retriever model.
Multi-query retrieval (MQR) algorithm, which effectively handles multiple
queries related to a specific topic, is applied to construct the Customized
dataset. Subsequently, we fine-tune a classifier using the Customized dataset
to identify the topic. To demonstrate the efficacy of our proposed approach, we
conduct evaluations on both widely used classification benchmark datasets and
manually constructed datasets with 291 diverse topics, which simulate diverse
contents encountered in real-world applications. DRAFT shows competitive or
superior performance compared to baselines that use in-context learning, such
as GPT-3 175B and InstructGPT 175B, on few-shot topic classification tasks
despite having 177 times fewer parameters, demonstrating its effectiveness.