Deep Active Learning: A Reality Check
Edrina Gashi Jiankang Deng Ismail Elezi
We conduct a comprehensive evaluation of state-of-the-art deep active
learning methods. Surprisingly, under general settings, no single-model method
decisively outperforms entropy-based active learning, and some even fall short
of random sampling. We delve into overlooked aspects like starting budget,
budget step, and pretraining's impact, revealing their significance in
achieving superior results. Additionally, we extend our evaluation to other
tasks, exploring the active learning effectiveness in combination with
semi-supervised learning, and object detection. Our experiments provide
valuable insights and concrete recommendations for future active learning
studies. By uncovering the limitations of current methods and understanding the
impact of different experimental settings, we aim to inspire more efficient
training of deep learning models in real-world scenarios with limited
annotation budgets. This work contributes to advancing active learning's
efficacy in deep learning and empowers researchers to make informed decisions
when applying active learning to their tasks.