False Negative/Positive Control for SAM on Noisy Medical Images
Xing Yao Han Liu Dewei Hu Daiwei Lu Ange Lou Hao Li Ruining Deng Gabriel Arenas Baris Oguz Nadav Schwartz Brett C Byram Ipek Oguz
The Segment Anything Model (SAM) is a recently developed all-range foundation
model for image segmentation. It can use sparse manual prompts such as bounding
boxes to generate pixel-level segmentation in natural images but struggles in
medical images such as low-contrast, noisy ultrasound images. We propose a
refined test-phase prompt augmentation technique designed to improve SAM's
performance in medical image segmentation. The method couples multi-box prompt
augmentation and an aleatoric uncertainty-based false-negative (FN) and
false-positive (FP) correction (FNPC) strategy. We evaluate the method on two
ultrasound datasets and show improvement in SAM's performance and robustness to
inaccurate prompts, without the necessity for further training or tuning.
Moreover, we present the Single-Slice-to-Volume (SS2V) method, enabling 3D
pixel-level segmentation using only the bounding box annotation from a single
2D slice. Our results allow efficient use of SAM in even noisy, low-contrast
medical images. The source code will be released soon.