Multi-rater Prompting for Ambiguous Medical Image Segmentation
Jinhong Wang Yi Cheng Jintai Chen Hongxia Xu Danny Chen Jian Wu
Multi-rater annotations commonly occur when medical images are independently
annotated by multiple experts (raters). In this paper, we tackle two challenges
arisen in multi-rater annotations for medical image segmentation (called
ambiguous medical image segmentation): (1) How to train a deep learning model
when a group of raters produces a set of diverse but plausible annotations, and
(2) how to fine-tune the model efficiently when computation resources are not
available for re-training the entire model on a different dataset domain. We
propose a multi-rater prompt-based approach to address these two challenges
altogether. Specifically, we introduce a series of rater-aware prompts that can
be plugged into the U-Net model for uncertainty estimation to handle
multi-annotation cases. During the prompt-based fine-tuning process, only 0.3%
of learnable parameters are required to be updated comparing to training the
entire model. Further, in order to integrate expert consensus and disagreement,
we explore different multi-rater incorporation strategies and design a
mix-training strategy for comprehensive insight learning. Extensive experiments
verify the effectiveness of our new approach for ambiguous medical image
segmentation on two public datasets while alleviating the heavy burden of model
re-training.