SegFormer3D: an Efficient Transformer for 3D Medical Image Segmentation
Shehan Perera Pouyan Navard Alper Yilmaz
The adoption of Vision Transformers (ViTs) based architectures represents a
significant advancement in 3D Medical Image (MI) segmentation, surpassing
traditional Convolutional Neural Network (CNN) models by enhancing global
contextual understanding. While this paradigm shift has significantly enhanced
3D segmentation performance, state-of-the-art architectures require extremely
large and complex architectures with large scale computing resources for
training and deployment. Furthermore, in the context of limited datasets, often
encountered in medical imaging, larger models can present hurdles in both model
generalization and convergence. In response to these challenges and to
demonstrate that lightweight models are a valuable area of research in 3D
medical imaging, we present SegFormer3D, a hierarchical Transformer that
calculates attention across multiscale volumetric features. Additionally,
SegFormer3D avoids complex decoders and uses an all-MLP decoder to aggregate
local and global attention features to produce highly accurate segmentation
masks. The proposed memory efficient Transformer preserves the performance
characteristics of a significantly larger model in a compact design.
SegFormer3D democratizes deep learning for 3D medical image segmentation by
offering a model with 33x less parameters and a 13x reduction in GFLOPS
compared to the current state-of-the-art (SOTA). We benchmark SegFormer3D
against the current SOTA models on three widely used datasets Synapse, BRaTs,
and ACDC, achieving competitive results. Code:
https://github.com/OSUPCVLab/SegFormer3D.git