Generative Medical Segmentation
Jiayu Huo Xi Ouyang Sébastien Ourselin Rachel Sparks
Rapid advancements in medical image segmentation performance have been
significantly driven by the development of Convolutional Neural Networks (CNNs)
and Vision Transformers (ViTs). However, these models introduce high
computational demands and often have limited ability to generalize across
diverse medical imaging datasets. In this manuscript, we introduce Generative
Medical Segmentation (GMS), a novel approach leveraging a generative model for
image segmentation. Concretely, GMS employs a robust pre-trained Variational
Autoencoder (VAE) to derive latent representations of both images and masks,
followed by a mapping model that learns the transition from image to mask in
the latent space. This process culminates in generating a precise segmentation
mask within the image space using the pre-trained VAE decoder. The design of
GMS leads to fewer learnable parameters in the model, resulting in a reduced
computational burden and enhanced generalization capability. Our extensive
experimental analysis across five public datasets in different medical imaging
domains demonstrates GMS outperforms existing discriminative segmentation
models and has remarkable domain generalization. Our experiments suggest GMS
could set a new benchmark for medical image segmentation, offering a scalable
and effective solution. GMS implementation and model weights are available at
https://github.com/King-HAW/GMS.