EMIT-Diff: Enhancing Medical Image Segmentation via Text-Guided
  Diffusion Model
Zheyuan Zhang Lanhong Yao Bin Wang Debesh Jha Elif Keles Alpay Medetalibeyoglu Ulas Bagci
Large-scale, big-variant, and high-quality data are crucial for developing
robust and successful deep-learning models for medical applications since they
potentially enable better generalization performance and avoid overfitting.
However, the scarcity of high-quality labeled data always presents significant
challenges. This paper proposes a novel approach to address this challenge by
developing controllable diffusion models for medical image synthesis, called
EMIT-Diff. We leverage recent diffusion probabilistic models to generate
realistic and diverse synthetic medical image data that preserve the essential
characteristics of the original medical images by incorporating edge
information of objects to guide the synthesis process. In our approach, we
ensure that the synthesized samples adhere to medically relevant constraints
and preserve the underlying structure of imaging data. Due to the random
sampling process by the diffusion model, we can generate an arbitrary number of
synthetic images with diverse appearances. To validate the effectiveness of our
proposed method, we conduct an extensive set of medical image segmentation
experiments on multiple datasets, including Ultrasound breast (+13.87%), CT
spleen (+0.38%), and MRI prostate (+7.78%), achieving significant improvements
over the baseline segmentation methods. For the first time, to our best
knowledge, the promising results demonstrate the effectiveness of our EMIT-Diff
for medical image segmentation tasks and show the feasibility of introducing a
first-ever text-guided diffusion model for general medical image segmentation
tasks. With carefully designed ablation experiments, we investigate the
influence of various data augmentation ratios, hyper-parameter settings, patch
size for generating random merging mask settings, and combined influence with
different network architectures.