Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport
Wentong Li Yuqian Yuan Song Wang Jianke Zhu Jianshu Li Jian Liu Lei Zhang
Weakly-supervised image segmentation has recently attracted increasing
research attentions, aiming to avoid the expensive pixel-wise labeling. In this
paper, we present an effective method, namely Point2Mask, to achieve
high-quality panoptic prediction using only a single random point annotation
per target for training. Specifically, we formulate the panoptic pseudo-mask
generation as an Optimal Transport (OT) problem, where each ground-truth (gt)
point label and pixel sample are defined as the label supplier and consumer,
respectively. The transportation cost is calculated by the introduced
task-oriented maps, which focus on the category-wise and instance-wise
differences among the various thing and stuff targets. Furthermore, a
centroid-based scheme is proposed to set the accurate unit number for each gt
point supplier. Hence, the pseudo-mask generation is converted into finding the
optimal transport plan at a globally minimal transportation cost, which can be
solved via the Sinkhorn-Knopp Iteration. Experimental results on Pascal VOC and
COCO demonstrate the promising performance of our proposed Point2Mask approach
to point-supervised panoptic segmentation. Source code is available at:
https://github.com/LiWentomng/Point2Mask.