SAM-Med2D
Junlong Cheng Jin Ye Zhongying Deng Jianpin Chen Tianbin Li Haoyu Wang Yanzhou Su Ziyan Huang Jilong Chen Lei Jiang Hui Sun Junjun He Shaoting Zhang Min Zhu Yu Qiao
The Segment Anything Model (SAM) represents a state-of-the-art research
advancement in natural image segmentation, achieving impressive results with
input prompts such as points and bounding boxes. However, our evaluation and
recent research indicate that directly applying the pretrained SAM to medical
image segmentation does not yield satisfactory performance. This limitation
primarily arises from significant domain gap between natural images and medical
images. To bridge this gap, we introduce SAM-Med2D, the most comprehensive
studies on applying SAM to medical 2D images. Specifically, we first collect
and curate approximately 4.6M images and 19.7M masks from public and private
datasets, constructing a large-scale medical image segmentation dataset
encompassing various modalities and objects. Then, we comprehensively fine-tune
SAM on this dataset and turn it into SAM-Med2D. Unlike previous methods that
only adopt bounding box or point prompts as interactive segmentation approach,
we adapt SAM to medical image segmentation through more comprehensive prompts
involving bounding boxes, points, and masks. We additionally fine-tune the
encoder and decoder of the original SAM to obtain a well-performed SAM-Med2D,
leading to the most comprehensive fine-tuning strategies to date. Finally, we
conducted a comprehensive evaluation and analysis to investigate the
performance of SAM-Med2D in medical image segmentation across various
modalities, anatomical structures, and organs. Concurrently, we validated the
generalization capability of SAM-Med2D on 9 datasets from MICCAI 2023
challenge. Overall, our approach demonstrated significantly superior
performance and generalization capability compared to SAM.