Dual self-distillation of U-shaped networks for 3D medical image
  segmentation
Soumyanil Banerjee Ming Dong Carri Glide-Hurst
U-shaped networks and its variants have demonstrated exceptional results for
medical image segmentation. In this paper, we propose a novel dual
self-distillation (DSD) framework for U-shaped networks for 3D medical image
segmentation. DSD distills knowledge from the ground-truth segmentation labels
to the decoder layers and also between the encoder and decoder layers of a
single U-shaped network. DSD is a generalized training strategy that could be
attached to the backbone architecture of any U-shaped network to further
improve its segmentation performance. We attached DSD on two state-of-the-art
U-shaped backbones, and extensive experiments on two public 3D medical image
segmentation datasets (cardiac substructure and brain tumor) demonstrated
significant improvement over those backbones. On average, after attaching DSD
to the U-shaped backbones, we observed an improvement of 4.25% and 3.15% in
Dice similarity score for cardiac substructure and brain tumor segmentation
respectively.