Segment Anything Model Can Not Segment Anything: Assessing AI Foundation
  Model's Generalizability in Permafrost Mapping
Wenwen Li Chia-Yu Hsu Sizhe Wang Yezhou Yang Hyunho Lee Anna Liljedahl Chandi Witharana Yili Yang Brendan M. Rogers Samantha T. Arundel Matthew B. Jones Kenton McHenry Patricia Solis
This paper assesses trending AI foundation models, especially emerging
computer vision foundation models and their performance in natural landscape
feature segmentation. While the term foundation model has quickly garnered
interest from the geospatial domain, its definition remains vague. Hence, this
paper will first introduce AI foundation models and their defining
characteristics. Built upon the tremendous success achieved by Large Language
Models (LLMs) as the foundation models for language tasks, this paper discusses
the challenges of building foundation models for geospatial artificial
intelligence (GeoAI) vision tasks. To evaluate the performance of large AI
vision models, especially Meta's Segment Anything Model (SAM), we implemented
different instance segmentation pipelines that minimize the changes to SAM to
leverage its power as a foundation model. A series of prompt strategies was
developed to test SAM's performance regarding its theoretical upper bound of
predictive accuracy, zero-shot performance, and domain adaptability through
fine-tuning. The analysis used two permafrost feature datasets, ice-wedge
polygons and retrogressive thaw slumps because (1) these landform features are
more challenging to segment than manmade features due to their complicated
formation mechanisms, diverse forms, and vague boundaries; (2) their presence
and changes are important indicators for Arctic warming and climate change. The
results show that although promising, SAM still has room for improvement to
support AI-augmented terrain mapping. The spatial and domain generalizability
of this finding is further validated using a more general dataset EuroCrop for
agricultural field mapping. Finally, we discuss future research directions that
strengthen SAM's applicability in challenging geospatial domains.