AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene
  Segmentation
Jay N. Paranjape Nithin Gopalakrishnan Nair Shameema Sikder S. Swaroop Vedula Vishal M. Patel
Segmentation is a fundamental problem in surgical scene analysis using
artificial intelligence. However, the inherent data scarcity in this domain
makes it challenging to adapt traditional segmentation techniques for this
task. To tackle this issue, current research employs pretrained models and
finetunes them on the given data. Even so, these require training deep networks
with millions of parameters every time new data becomes available. A recently
published foundation model, Segment-Anything (SAM), generalizes well to a large
variety of natural images, hence tackling this challenge to a reasonable
extent. However, SAM does not generalize well to the medical domain as is
without utilizing a large amount of compute resources for fine-tuning and using
task-specific prompts. Moreover, these prompts are in the form of
bounding-boxes or foreground/background points that need to be annotated
explicitly for every image, making this solution increasingly tedious with
higher data size. In this work, we propose AdaptiveSAM - an adaptive
modification of SAM that can adjust to new datasets quickly and efficiently,
while enabling text-prompted segmentation. For finetuning AdaptiveSAM, we
propose an approach called bias-tuning that requires a significantly smaller
number of trainable parameters than SAM (less than 2\%). At the same time,
AdaptiveSAM requires negligible expert intervention since it uses free-form
text as prompt and can segment the object of interest with just the label name
as prompt. Our experiments show that AdaptiveSAM outperforms current
state-of-the-art methods on various medical imaging datasets including surgery,
ultrasound and X-ray. Code is available at
https://github.com/JayParanjape/biastuning