Learnable Prompt for Few-Shot Semantic Segmentation in Remote Sensing
  Domain
Steve Andreas Immanuel Hagai Raja Sinulingga
Few-shot segmentation is a task to segment objects or regions of novel
classes within an image given only a few annotated examples. In the generalized
setting, the task extends to segment both the base and the novel classes. The
main challenge is how to train the model such that the addition of novel
classes does not hurt the base classes performance, also known as catastrophic
forgetting. To mitigate this issue, we use SegGPT as our base model and train
it on the base classes. Then, we use separate learnable prompts to handle
predictions for each novel class. To handle various object sizes which
typically present in remote sensing domain, we perform patch-based prediction.
To address the discontinuities along patch boundaries, we propose a
patch-and-stitch technique by re-framing the problem as an image inpainting
task. During inference, we also utilize image similarity search over image
embeddings for prompt selection and novel class filtering to reduce false
positive predictions. Based on our experiments, our proposed method boosts the
weighted mIoU of a simple fine-tuned SegGPT from 15.96 to 35.08 on the
validation set of few-shot OpenEarthMap dataset given in the challenge.