Influence of Prompting Strategies on Segment Anything Model (SAM) for
  Short-axis Cardiac MRI segmentation
Josh Stein Maxime Di Folco Julia A. Schnabel
The Segment Anything Model (SAM) has recently emerged as a significant
breakthrough in foundation models, demonstrating remarkable zero-shot
performance in object segmentation tasks. While SAM is designed for
generalization, it exhibits limitations in handling specific medical imaging
tasks that require fine-structure segmentation or precise boundaries. In this
paper, we focus on the task of cardiac magnetic resonance imaging (cMRI)
short-axis view segmentation using the SAM foundation model. We conduct a
comprehensive investigation of the impact of different prompting strategies
(including bounding boxes, positive points, negative points, and their
combinations) on segmentation performance. We evaluate on two public datasets
using the baseline model and models fine-tuned with varying amounts of
annotated data, ranging from a limited number of volumes to a fully annotated
dataset. Our findings indicate that prompting strategies significantly
influence segmentation performance. Combining positive points with either
bounding boxes or negative points shows substantial benefits, but little to no
benefit when combined simultaneously. We further observe that fine-tuning SAM
with a few annotated volumes improves segmentation performance when properly
prompted. Specifically, fine-tuning with bounding boxes has a positive impact,
while fine-tuning without bounding boxes leads to worse results compared to
baseline.