On the Query Strategies for Efficient Online Active Distillation
Michele Boldo Enrico Martini Mirco De Marchi Stefano Aldegheri Nicola Bombieri
Deep Learning (DL) requires lots of time and data, resulting in high
computational demands. Recently, researchers employ Active Learning (AL) and
online distillation to enhance training efficiency and real-time model
adaptation. This paper evaluates a set of query strategies to achieve the best
training results. It focuses on Human Pose Estimation (HPE) applications,
assessing the impact of selected frames during training using two approaches: a
classical offline method and a online evaluation through a continual learning
approach employing knowledge distillation, on a popular state-of-the-art HPE
dataset. The paper demonstrates the possibility of enabling training at the
edge lightweight models, adapting them effectively to new contexts in
real-time.