Guided Prompting in SAM for Weakly Supervised Cell Segmentation in
  Histopathological Images
Aayush Kumar Tyagi Vaibhav Mishra Prathosh A. P. Mausam
Cell segmentation in histopathological images plays a crucial role in
understanding, diagnosing, and treating many diseases. However, data annotation
for this is expensive since there can be a large number of cells per image, and
expert pathologists are needed for labelling images. Instead, our paper focuses
on using weak supervision -- annotation from related tasks -- to induce a
segmenter. Recent foundation models, such as Segment Anything (SAM), can use
prompts to leverage additional supervision during inference. SAM has performed
remarkably well in natural image segmentation tasks; however, its applicability
to cell segmentation has not been explored.
  In response, we investigate guiding the prompting procedure in SAM for weakly
supervised cell segmentation when only bounding box supervision is available.
We develop two workflows: (1) an object detector's output as a test-time prompt
to SAM (D-SAM), and (2) SAM as pseudo mask generator over training data to
train a standalone segmentation model (SAM-S). On finding that both workflows
have some complementary strengths, we develop an integer programming-based
approach to reconcile the two sets of segmentation masks, achieving yet higher
performance. We experiment on three publicly available cell segmentation
datasets namely, ConSep, MoNuSeg, and TNBC, and find that all SAM-based
solutions hugely outperform existing weakly supervised image segmentation
models, obtaining 9-15 pt Dice gains.