STARS: Zero-shot Sim-to-Real Transfer for Segmentation of Shipwrecks in
  Sonar Imagery
Advaith Venkatramanan Sethuraman Katherine A. Skinner
In this paper, we address the problem of sim-to-real transfer for object
segmentation when there is no access to real examples of an object of interest
during training, i.e. zero-shot sim-to-real transfer for segmentation. We focus
on the application of shipwreck segmentation in side scan sonar imagery. Our
novel segmentation network, STARS, addresses this challenge by fusing a
predicted deformation field and anomaly volume, allowing it to generalize
better to real sonar images and achieve more effective zero-shot sim-to-real
transfer for image segmentation. We evaluate the sim-to-real transfer
capabilities of our method on a real, expert-labeled side scan sonar dataset of
shipwrecks collected from field work surveys with an autonomous underwater
vehicle (AUV). STARS is trained entirely in simulation and performs zero-shot
shipwreck segmentation with no additional fine-tuning on real data. Our method
provides a significant 20% increase in segmentation performance for the
targeted shipwreck class compared to the best baseline.