Multi-level Asymmetric Contrastive Learning for Medical Image
  Segmentation Pre-training
Shuang Zeng Lei Zhu Xinliang Zhang Zifeng Tian Qian Chen Lujia Jin Jiayi Wang Yanye Lu
Contrastive learning, which is a powerful technique for learning image-level
representations from unlabeled data, leads a promising direction to dealing
with the dilemma between large-scale pre-training and limited labeled data.
However, most existing contrastive learning strategies are designed mainly for
downstream tasks of natural images, therefore they are sub-optimal and even
worse than learning from scratch when directly applied to medical images whose
downstream tasks are usually segmentation. In this work, we propose a novel
asymmetric contrastive learning framework named JCL for medical image
segmentation with self-supervised pre-training. Specifically, (1) A novel
asymmetric contrastive learning strategy is proposed to pre-train both encoder
and decoder simultaneously in one-stage to provide better initialization for
segmentation models. (2) A multi-level contrastive loss is designed to take the
correspondence among feature-level, image-level and pixel-level projections,
respectively into account to make sure multi-level representations can be
learned by the encoder and decoder during pre-training. (3) Experiments on
multiple medical image datasets indicate our JCL framework outperforms existing
SOTA contrastive learning strategies.