Self-Knowledge Distillation for Surgical Phase Recognition
Jinglu Zhang Santiago Barbarisi Abdolrahim Kadkhodamohammadi Danail Stoyanov Imanol Luengo
Purpose: Advances in surgical phase recognition are generally led by training
deeper networks. Rather than going further with a more complex solution, we
believe that current models can be exploited better. We propose a
self-knowledge distillation framework that can be integrated into current
state-of-the-art (SOTA) models without requiring any extra complexity to the
models or annotations.
  Methods: Knowledge distillation is a framework for network regularization
where knowledge is distilled from a teacher network to a student network. In
self-knowledge distillation, the student model becomes the teacher such that
the network learns from itself. Most phase recognition models follow an
encoder-decoder framework. Our framework utilizes self-knowledge distillation
in both stages. The teacher model guides the training process of the student
model to extract enhanced feature representations from the encoder and build a
more robust temporal decoder to tackle the over-segmentation problem.
  Results: We validate our proposed framework on the public dataset Cholec80.
Our framework is embedded on top of four popular SOTA approaches and
consistently improves their performance. Specifically, our best GRU model
boosts performance by +3.33% accuracy and +3.95% F1-score over the same
baseline model.
  Conclusion: We embed a self-knowledge distillation framework for the first
time in the surgical phase recognition training pipeline. Experimental results
demonstrate that our simple yet powerful framework can improve performance of
existing phase recognition models. Moreover, our extensive experiments show
that even with 75% of the training set we still achieve performance on par with
the same baseline model trained on the full set.