Querying Easily Flip-flopped Samples for Deep Active Learning
Seong Jin Cho Gwangsu Kim Junghyun Lee Jinwoo Shin Chang D. Yoo
Active learning is a machine learning paradigm that aims to improve the
performance of a model by strategically selecting and querying unlabeled data.
One effective selection strategy is to base it on the model's predictive
uncertainty, which can be interpreted as a measure of how informative a sample
is. The sample's distance to the decision boundary is a natural measure of
predictive uncertainty, but it is often intractable to compute, especially for
complex decision boundaries formed in multiclass classification tasks. To
address this issue, this paper proposes the {\it least disagree metric} (LDM),
defined as the smallest probability of disagreement of the predicted label, and
an estimator for LDM proven to be asymptotically consistent under mild
assumptions. The estimator is computationally efficient and can be easily
implemented for deep learning models using parameter perturbation. The
LDM-based active learning is performed by querying unlabeled data with the
smallest LDM. Experimental results show that our LDM-based active learning
algorithm obtains state-of-the-art overall performance on all considered
datasets and deep architectures.