Task Specific Pretraining with Noisy Labels for Remote sensing Image
  Segmentation
Chenying Liu Conrad Albrecht Yi Wang Xiao Xiang Zhu
In recent years, self-supervision has drawn a lot of attention in remote
sensing society due to its ability to reduce the demand of exact labels in
supervised deep learning model training. Self-supervision methods generally
utilize image-level information to pretrain models in an unsupervised fashion.
Though these pretrained encoders show effectiveness in many downstream tasks,
their performance on segmentation tasks is often not as good as that on
classification tasks. On the other hand, many easily available label sources
(e.g., automatic labeling tools and land cover land use products) exist, which
can provide a large amount of noisy labels for segmentation model training. In
this work, we propose to explore the under-exploited potential of noisy labels
for segmentation task specific pretraining, and exam its robustness when
confronted with mismatched categories and different decoders during
fine-tuning. Specifically, we inspect the impacts of noisy labels on different
layers in supervised model training to serve as the basis of our work.
Experiments on two datasets indicate the effectiveness of task specific
supervised pretraining with noisy labels. The findings are expected to shed
light on new avenues for improving the accuracy and versatility of pretraining
strategies for remote sensing image segmentation.