Attention-guided Feature Distillation for Semantic Segmentation
Amir M. Mansourian Arya Jalali Rozhan Ahmadi Shohreh Kasaei
In contrast to existing complex methodologies commonly employed for
distilling knowledge from a teacher to a student, the pro-posed method
showcases the efficacy of a simple yet powerful method for utilizing refined
feature maps to transfer attention. The proposed method has proven to be
effective in distilling rich information, outperforming existing methods in
semantic segmentation as a dense prediction task. The proposed Attention-guided
Feature Distillation (AttnFD) method, em-ploys the Convolutional Block
Attention Module (CBAM), which refines feature maps by taking into account both
channel-specific and spatial information content. By only using the Mean
Squared Error (MSE) loss function between the refined feature maps of the
teacher and the student,AttnFD demonstrates outstanding performance in semantic
segmentation, achieving state-of-the-art results in terms of mean Intersection
over Union (mIoU) on the PascalVoc 2012 and Cityscapes datasets. The Code is
available at https://github.com/AmirMansurian/AttnFD.