Self-Supervised Vision Transformers Are Efficient Segmentation Learners
  for Imperfect Labels
Seungho Lee Seoungyoon Kang Hyunjung Shim
This study demonstrates a cost-effective approach to semantic segmentation
using self-supervised vision transformers (SSVT). By freezing the SSVT backbone
and training a lightweight segmentation head, our approach effectively utilizes
imperfect labels, thereby improving robustness to label imperfections.
Empirical experiments show significant performance improvements over existing
methods for various annotation types, including scribble, point-level, and
image-level labels. The research highlights the effectiveness of
self-supervised vision transformers in dealing with imperfect labels, providing
a practical and efficient solution for semantic segmentation while reducing
annotation costs. Through extensive experiments, we confirm that our method
outperforms baseline models for all types of imperfect labels. Especially under
the zero-shot vision-language-model-based label, our model exhibits 11.5\%p
performance gain compared to the baseline.