What is Point Supervision Worth in Video Instance Segmentation?
Shuaiyi Huang De-An Huang Zhiding Yu Shiyi Lan Subhashree Radhakrishnan Jose M. Alvarez Abhinav Shrivastava Anima Anandkumar
Video instance segmentation (VIS) is a challenging vision task that aims to
detect, segment, and track objects in videos. Conventional VIS methods rely on
densely-annotated object masks which are expensive. We reduce the human
annotations to only one point for each object in a video frame during training,
and obtain high-quality mask predictions close to fully supervised models. Our
proposed training method consists of a class-agnostic proposal generation
module to provide rich negative samples and a spatio-temporal point-based
matcher to match the object queries with the provided point annotations.
Comprehensive experiments on three VIS benchmarks demonstrate competitive
performance of the proposed framework, nearly matching fully supervised
methods.