Triple-View Knowledge Distillation for Semi-Supervised Semantic
  Segmentation
Ping Li Junjie Chen Li Yuan Xianghua Xu Mingli Song
To alleviate the expensive human labeling, semi-supervised semantic
segmentation employs a few labeled images and an abundant of unlabeled images
to predict the pixel-level label map with the same size. Previous methods often
adopt co-training using two convolutional networks with the same architecture
but different initialization, which fails to capture the sufficiently diverse
features. This motivates us to use tri-training and develop the triple-view
encoder to utilize the encoders with different architectures to derive diverse
features, and exploit the knowledge distillation skill to learn the
complementary semantics among these encoders. Moreover, existing methods simply
concatenate the features from both encoder and decoder, resulting in redundant
features that require large memory cost. This inspires us to devise a
dual-frequency decoder that selects those important features by projecting the
features from the spatial domain to the frequency domain, where the
dual-frequency channel attention mechanism is introduced to model the feature
importance. Therefore, we propose a Triple-view Knowledge Distillation
framework, termed TriKD, for semi-supervised semantic segmentation, including
the triple-view encoder and the dual-frequency decoder. Extensive experiments
were conducted on two benchmarks, \ie, Pascal VOC 2012 and Cityscapes, whose
results verify the superiority of the proposed method with a good tradeoff
between precision and inference speed.