Foundation Models for Biomedical Image Segmentation: A Survey
Ho Hin Lee Yu Gu Theodore Zhao Yanbo Xu Jianwei Yang Naoto Usuyama Cliff Wong Mu Wei Bennett A. Landman Yuankai Huo Alberto Santamaria-Pang Hoifung Poon
Recent advancements in biomedical image analysis have been significantly
driven by the Segment Anything Model (SAM). This transformative technology,
originally developed for general-purpose computer vision, has found rapid
application in medical image processing. Within the last year, marked by over
100 publications, SAM has demonstrated its prowess in zero-shot learning
adaptations for medical imaging. The fundamental premise of SAM lies in its
capability to segment or identify objects in images without prior knowledge of
the object type or imaging modality. This approach aligns well with tasks
achievable by the human visual system, though its application in non-biological
vision contexts remains more theoretically challenging. A notable feature of
SAM is its ability to adjust segmentation according to a specified resolution
scale or area of interest, akin to semantic priming. This adaptability has
spurred a wave of creativity and innovation in applying SAM to medical imaging.
Our review focuses on the period from April 1, 2023, to September 30, 2023, a
critical first six months post-initial publication. We examine the adaptations
and integrations of SAM necessary to address longstanding clinical challenges,
particularly in the context of 33 open datasets covered in our analysis. While
SAM approaches or achieves state-of-the-art performance in numerous
applications, it falls short in certain areas, such as segmentation of the
carotid artery, adrenal glands, optic nerve, and mandible bone. Our survey
delves into the innovative techniques where SAM's foundational approach excels
and explores the core concepts in translating and applying these models
effectively in diverse medical imaging scenarios.