Beyond Self-Attention: Deformable Large Kernel Attention for Medical
  Image Segmentation
Reza Azad Leon Niggemeier Michael Huttemann Amirhossein Kazerouni Ehsan Khodapanah Aghdam Yury Velichko Ulas Bagci Dorit Merhof
Medical image segmentation has seen significant improvements with transformer
models, which excel in grasping far-reaching contexts and global contextual
information. However, the increasing computational demands of these models,
proportional to the squared token count, limit their depth and resolution
capabilities. Most current methods process D volumetric image data
slice-by-slice (called pseudo 3D), missing crucial inter-slice information and
thus reducing the model's overall performance. To address these challenges, we
introduce the concept of \textbf{Deformable Large Kernel Attention (D-LKA
Attention)}, a streamlined attention mechanism employing large convolution
kernels to fully appreciate volumetric context. This mechanism operates within
a receptive field akin to self-attention while sidestepping the computational
overhead. Additionally, our proposed attention mechanism benefits from
deformable convolutions to flexibly warp the sampling grid, enabling the model
to adapt appropriately to diverse data patterns. We designed both 2D and 3D
adaptations of the D-LKA Attention, with the latter excelling in cross-depth
data understanding. Together, these components shape our novel hierarchical
Vision Transformer architecture, the \textit{D-LKA Net}. Evaluations of our
model against leading methods on popular medical segmentation datasets
(Synapse, NIH Pancreas, and Skin lesion) demonstrate its superior performance.
Our code implementation is publicly available at the:
https://github.com/mindflow-institue/deformableLKA