MOSformer: Momentum encoder-based inter-slice fusion transformer for
  medical image segmentation
De-Xing Huang Xiao-Hu Zhou Xiao-Liang Xie Shi-Qi Liu Zhen-Qiu Feng Mei-Jiang Gui Hao Li Tian-Yu Xiang Xiu-Ling Liu Zeng-Guang Hou
Medical image segmentation takes an important position in various clinical
applications. Deep learning has emerged as the predominant solution for
automated segmentation of volumetric medical images. 2.5D-based segmentation
models bridge computational efficiency of 2D-based models and spatial
perception capabilities of 3D-based models. However, prevailing 2.5D-based
models often treat each slice equally, failing to effectively learn and exploit
inter-slice information, resulting in suboptimal segmentation performances. In
this paper, a novel Momentum encoder-based inter-slice fusion transformer
(MOSformer) is proposed to overcome this issue by leveraging inter-slice
information at multi-scale feature maps extracted by different encoders.
Specifically, dual encoders are employed to enhance feature distinguishability
among different slices. One of the encoders is moving-averaged to maintain the
consistency of slice representations. Moreover, an IF-Swin transformer module
is developed to fuse inter-slice multi-scale features. The MOSformer is
evaluated on three benchmark datasets (Synapse, ACDC, and AMOS), establishing a
new state-of-the-art with 85.63%, 92.19%, and 85.43% of DSC, respectively.
These promising results indicate its competitiveness in medical image
segmentation. Codes and models of MOSformer will be made publicly available
upon acceptance.