Multiscale Progressive Text Prompt Network for Medical Image
  Segmentation
Xianjun Han Qianqian Chen Zhaoyang Xie Xuejun Li Hongyu Yang
The accurate segmentation of medical images is a crucial step in obtaining
reliable morphological statistics. However, training a deep neural network for
this task requires a large amount of labeled data to ensure high-accuracy
results. To address this issue, we propose using progressive text prompts as
prior knowledge to guide the segmentation process. Our model consists of two
stages. In the first stage, we perform contrastive learning on natural images
to pretrain a powerful prior prompt encoder (PPE). This PPE leverages text
prior prompts to generate multimodality features. In the second stage, medical
image and text prior prompts are sent into the PPE inherited from the first
stage to achieve the downstream medical image segmentation task. A multiscale
feature fusion block (MSFF) combines the features from the PPE to produce
multiscale multimodality features. These two progressive features not only
bridge the semantic gap but also improve prediction accuracy. Finally, an
UpAttention block refines the predicted results by merging the image and text
features. This design provides a simple and accurate way to leverage multiscale
progressive text prior prompts for medical image segmentation. Compared with
using only images, our model achieves high-quality results with low data
annotation costs. Moreover, our model not only has excellent reliability and
validity on medical images but also performs well on natural images. The
experimental results on different image datasets demonstrate that our model is
effective and robust for image segmentation.