Harmonized Spatial and Spectral Learning for Robust and Generalized
  Medical Image Segmentation
Vandan Gorade Sparsh Mittal Debesh Jha Rekha Singhal Ulas Bagci
Deep learning has demonstrated remarkable achievements in medical image
segmentation. However, prevailing deep learning models struggle with poor
generalization due to (i) intra-class variations, where the same class appears
differently in different samples, and (ii) inter-class independence, resulting
in difficulties capturing intricate relationships between distinct objects,
leading to higher false negative cases. This paper presents a novel approach
that synergies spatial and spectral representations to enhance
domain-generalized medical image segmentation. We introduce the innovative
Spectral Correlation Coefficient objective to improve the model's capacity to
capture middle-order features and contextual long-range dependencies. This
objective complements traditional spatial objectives by incorporating valuable
spectral information. Extensive experiments reveal that optimizing this
objective with existing architectures like UNet and TransUNet significantly
enhances generalization, interpretability, and noise robustness, producing more
confident predictions. For instance, in cardiac segmentation, we observe a 0.81
pp and 1.63 pp (pp = percentage point) improvement in DSC over UNet and
TransUNet, respectively. Our interpretability study demonstrates that, in most
tasks, objectives optimized with UNet outperform even TransUNet by introducing
global contextual information alongside local details. These findings
underscore the versatility and effectiveness of our proposed method across
diverse imaging modalities and medical domains.