PMFSNet: Polarized Multi-scale Feature Self-attention Network For
  Lightweight Medical Image Segmentation
Jiahui Zhong Wenhong Tian Yuanlun Xie Zhijia Liu Jie Ou Taoran Tian Lei Zhang
Current state-of-the-art medical image segmentation methods prioritize
accuracy but often at the expense of increased computational demands and larger
model sizes. Applying these large-scale models to the relatively limited scale
of medical image datasets tends to induce redundant computation, complicating
the process without the necessary benefits. This approach not only adds
complexity but also presents challenges for the integration and deployment of
lightweight models on edge devices. For instance, recent transformer-based
models have excelled in 2D and 3D medical image segmentation due to their
extensive receptive fields and high parameter count. However, their
effectiveness comes with a risk of overfitting when applied to small datasets
and often neglects the vital inductive biases of Convolutional Neural Networks
(CNNs), essential for local feature representation. In this work, we propose
PMFSNet, a novel medical imaging segmentation model that effectively balances
global and local feature processing while avoiding the computational redundancy
typical in larger models. PMFSNet streamlines the UNet-based hierarchical
structure and simplifies the self-attention mechanism's computational
complexity, making it suitable for lightweight applications. It incorporates a
plug-and-play PMFS block, a multi-scale feature enhancement module based on
attention mechanisms, to capture long-term dependencies. Extensive
comprehensive results demonstrate that even with a model (less than 1 million
parameters), our method achieves superior performance in various segmentation
tasks across different data scales. It achieves (IoU) metrics of 84.68%,
82.02%, and 78.82% on public datasets of teeth CT (CBCT), ovarian tumors
ultrasound(MMOTU), and skin lesions dermoscopy images (ISIC 2018),
respectively. The source code is available at
https://github.com/yykzjh/PMFSNet.