UVOSAM: A Mask-free Paradigm for Unsupervised Video Object Segmentation
  via Segment Anything Model
Zhenghao Zhang Zhichao Wei Shengfan Zhang Zuozhuo Dai Siyu Zhu
Unsupervised video object segmentation has made significant progress in
recent years, but the manual annotation of video mask datasets is expensive and
limits the diversity of available datasets. The Segment Anything Model (SAM)
has introduced a new prompt-driven paradigm for image segmentation, unlocking a
range of previously unexplored capabilities. In this paper, we propose a novel
paradigm called UVOSAM, which leverages SAM for unsupervised video object
segmentation without requiring video mask labels. To address SAM's limitations
in instance discovery and identity association, we introduce a video salient
object tracking network that automatically generates trajectories for prominent
foreground objects. These trajectories then serve as prompts for SAM to produce
video masks on a frame-by-frame basis. Our experimental results demonstrate
that UVOSAM significantly outperforms current mask-supervised methods. These
findings suggest that UVOSAM has the potential to improve unsupervised video
object segmentation and reduce the cost of manual annotation.