Segment Any Cell: A SAM-based Auto-prompting Fine-tuning Framework for
  Nuclei Segmentation
Saiyang Na Yuzhi Guo Feng Jiang Hehuan Ma Junzhou Huang
In the rapidly evolving field of AI research, foundational models like BERT
and GPT have significantly advanced language and vision tasks. The advent of
pretrain-prompting models such as ChatGPT and Segmentation Anything Model (SAM)
has further revolutionized image segmentation. However, their applications in
specialized areas, particularly in nuclei segmentation within medical imaging,
reveal a key challenge: the generation of high-quality, informative prompts is
as crucial as applying state-of-the-art (SOTA) fine-tuning techniques on
foundation models. To address this, we introduce Segment Any Cell (SAC), an
innovative framework that enhances SAM specifically for nuclei segmentation.
SAC integrates a Low-Rank Adaptation (LoRA) within the attention layer of the
Transformer to improve the fine-tuning process, outperforming existing SOTA
methods. It also introduces an innovative auto-prompt generator that produces
effective prompts to guide segmentation, a critical factor in handling the
complexities of nuclei segmentation in biomedical imaging. Our extensive
experiments demonstrate the superiority of SAC in nuclei segmentation tasks,
proving its effectiveness as a tool for pathologists and researchers. Our
contributions include a novel prompt generation strategy, automated
adaptability for diverse segmentation tasks, the innovative application of
Low-Rank Attention Adaptation in SAM, and a versatile framework for semantic
segmentation challenges.