MS-Net: A Multi-modal Self-supervised Network for Fine-Grained
  Classification of Aircraft in SAR Images
Bingying Yue Jianhao Li Hao Shi Yupei Wang Honghu Zhong
Synthetic aperture radar (SAR) imaging technology is commonly used to provide
24-hour all-weather earth observation. However, it still has some drawbacks in
SAR target classification, especially in fine-grained classification of
aircraft: aircrafts in SAR images have large intra-class diversity and
inter-class similarity; the number of effective samples is insufficient and
it's hard to annotate. To address these issues, this article proposes a novel
multi-modal self-supervised network (MS-Net) for fine-grained classification of
aircraft. Firstly, in order to entirely exploit the potential of multi-modal
information, a two-sided path feature extraction network (TSFE-N) is
constructed to enhance the image feature of the target and obtain the domain
knowledge feature of text mode. Secondly, a contrastive self-supervised
learning (CSSL) framework is employed to effectively learn useful
label-independent feature from unbalanced data, a similarity per-ception loss
(SPloss) is proposed to avoid network overfitting. Finally, TSFE-N is used as
the encoder of CSSL to obtain the classification results. Through a large
number of experiments, our MS-Net can effectively reduce the difficulty of
classifying similar types of aircrafts. In the case of no label, the proposed
algorithm achieves an accuracy of 88.46% for 17 types of air-craft
classification task, which has pioneering significance in the field of
fine-grained classification of aircraft in SAR images.