Task Integration Distillation for Object Detectors
Hai Su ZhenWen Jian Songsen Yu
Knowledge distillation is a widely adopted technique for model lightening.
However, the performance of most knowledge distillation methods in the domain
of object detection is not satisfactory. Typically, knowledge distillation
approaches consider only the classification task among the two sub-tasks of an
object detector, largely overlooking the regression task. This oversight leads
to a partial understanding of the object detector's comprehensive task,
resulting in skewed estimations and potentially adverse effects. Therefore, we
propose a knowledge distillation method that addresses both the classification
and regression tasks, incorporating a task significance strategy. By evaluating
the importance of features based on the output of the detector's two sub-tasks,
our approach ensures a balanced consideration of both classification and
regression tasks in object detection. Drawing inspiration from real-world
teaching processes and the definition of learning condition, we introduce a
method that focuses on both key and weak areas. By assessing the value of
features for knowledge distillation based on their importance differences, we
accurately capture the current model's learning situation. This method
effectively prevents the issue of biased predictions about the model's learning
reality caused by an incomplete utilization of the detector's outputs.