MobileUtr: Revisiting the relationship between light-weight CNN and
  Transformer for efficient medical image segmentation
Fenghe Tang Bingkun Nian Jianrui Ding Quan Quan Jie Yang Wei Liu S. Kevin Zhou
Due to the scarcity and specific imaging characteristics in medical images,
light-weighting Vision Transformers (ViTs) for efficient medical image
segmentation is a significant challenge, and current studies have not yet paid
attention to this issue. This work revisits the relationship between CNNs and
Transformers in lightweight universal networks for medical image segmentation,
aiming to integrate the advantages of both worlds at the infrastructure design
level. In order to leverage the inductive bias inherent in CNNs, we abstract a
Transformer-like lightweight CNNs block (ConvUtr) as the patch embeddings of
ViTs, feeding Transformer with denoised, non-redundant and highly condensed
semantic information. Moreover, an adaptive Local-Global-Local (LGL) block is
introduced to facilitate efficient local-to-global information flow exchange,
maximizing Transformer's global context information extraction capabilities.
Finally, we build an efficient medical image segmentation model (MobileUtr)
based on CNN and Transformer. Extensive experiments on five public medical
image datasets with three different modalities demonstrate the superiority of
MobileUtr over the state-of-the-art methods, while boasting lighter weights and
lower computational cost. Code is available at
https://github.com/FengheTan9/MobileUtr.