Bridging Diversity and Uncertainty in Active learning with
  Self-Supervised Pre-Training
Paul Doucet Benjamin Estermann Till Aczel Roger Wattenhofer
This study addresses the integration of diversity-based and uncertainty-based
sampling strategies in active learning, particularly within the context of
self-supervised pre-trained models. We introduce a straightforward heuristic
called TCM that mitigates the cold start problem while maintaining strong
performance across various data levels. By initially applying TypiClust for
diversity sampling and subsequently transitioning to uncertainty sampling with
Margin, our approach effectively combines the strengths of both strategies. Our
experiments demonstrate that TCM consistently outperforms existing methods
across various datasets in both low and high data regimes.