Rethinking Data Perturbation and Model Stabilization for Semi-supervised
  Medical Image Segmentation
Zhen Zhao Ye Liu Meng Zhao Di Yin Yixuan Yuan Luping Zhou
Studies on semi-supervised medical image segmentation (SSMIS) have seen fast
progress recently. Due to the limited labelled data, SSMIS methods mainly focus
on effectively leveraging unlabeled data to enhance the segmentation
performance. However, despite their promising performance, current
state-of-the-art methods often prioritize integrating complex techniques and
loss terms rather than addressing the core challenges of semi-supervised
scenarios directly. We argue that the key to SSMIS lies in generating
substantial and appropriate prediction disagreement on unlabeled data. To this
end, we emphasize the crutiality of data perturbation and model stabilization
in semi-supervised segmentation, and propose a simple yet effective approach to
boost SSMIS performance significantly, dubbed DPMS. Specifically, we first
revisit SSMIS from three distinct perspectives: the data, the model, and the
loss, and conduct a comprehensive study of corresponding strategies to examine
their effectiveness. Based on these examinations, we then propose DPMS, which
adopts a plain teacher-student framework with a standard supervised loss and
unsupervised consistency loss. To produce appropriate prediction disagreements,
DPMS perturbs the unlabeled data via strong augmentations to enlarge prediction
disagreements considerably. On the other hand, using EMA teacher when strong
augmentation is applied does not necessarily improve performance. DPMS further
utilizes a forwarding-twice and momentum updating strategies for normalization
statistics to stabilize the training on unlabeled data effectively. Despite its
simplicity, DPMS can obtain new state-of-the-art performance on the public 2D
ACDC and 3D LA datasets across various semi-supervised settings, e.g. obtaining
a remarkable 22.62% improvement against previous SOTA on ACDC with 5% labels.