FNOSeg3D: Resolution-Robust 3D Image Segmentation with Fourier Neural
  Operator
Ken C. L. Wong Hongzhi Wang Tanveer Syeda-Mahmood
Due to the computational complexity of 3D medical image segmentation,
training with downsampled images is a common remedy for out-of-memory errors in
deep learning. Nevertheless, as standard spatial convolution is sensitive to
variations in image resolution, the accuracy of a convolutional neural network
trained with downsampled images can be suboptimal when applied on the original
resolution. To address this limitation, we introduce FNOSeg3D, a 3D
segmentation model robust to training image resolution based on the Fourier
neural operator (FNO). The FNO is a deep learning framework for learning
mappings between functions in partial differential equations, which has the
appealing properties of zero-shot super-resolution and global receptive field.
We improve the FNO by reducing its parameter requirement and enhancing its
learning capability through residual connections and deep supervision, and
these result in our FNOSeg3D model which is parameter efficient and resolution
robust. When tested on the BraTS'19 dataset, it achieved superior robustness to
training image resolution than other tested models with less than 1% of their
model parameters.