Interactive Multi-fidelity Learning for Cost-effective Adaptation of
  Language Model with Sparse Human Supervision
Jiaxin Zhang Zhuohang Li Kamalika Das Sricharan Kumar
Large language models (LLMs) have demonstrated remarkable capabilities in
various tasks. However, their suitability for domain-specific tasks, is limited
due to their immense scale at deployment, susceptibility to misinformation, and
more importantly, high data annotation costs. We propose a novel Interactive
Multi-Fidelity Learning (IMFL) framework for the cost-effective development of
small domain-specific LMs under limited annotation budgets. Our approach
formulates the domain-specific fine-tuning process as a multi-fidelity learning
problem, focusing on identifying the optimal acquisition strategy that balances
between low-fidelity automatic LLM annotations and high-fidelity human
annotations to maximize model performance. We further propose an
exploration-exploitation query strategy that enhances annotation diversity and
informativeness, incorporating two innovative designs: 1) prompt retrieval that
selects in-context examples from human-annotated samples to improve LLM
annotation, and 2) variable batch size that controls the order for choosing
each fidelity to facilitate knowledge distillation, ultimately enhancing
annotation quality. Extensive experiments on financial and medical tasks
demonstrate that IMFL achieves superior performance compared with single
fidelity annotations. Given a limited budget of human annotation, IMFL
significantly outperforms the human annotation baselines in all four tasks and
achieves very close performance as human annotations on two of the tasks. These
promising results suggest that the high human annotation costs in
domain-specific tasks can be significantly reduced by employing IMFL, which
utilizes fewer human annotations, supplemented with cheaper and faster LLM
(e.g., GPT-3.5) annotations to achieve comparable performance.