When 3D Bounding-Box Meets SAM: Point Cloud Instance Segmentation with
  Weak-and-Noisy Supervision
Qingtao Yu Heming Du Chen Liu Xin Yu
Learning from bounding-boxes annotations has shown great potential in
weakly-supervised 3D point cloud instance segmentation. However, we observed
that existing methods would suffer severe performance degradation with
perturbed bounding box annotations. To tackle this issue, we propose a
complementary image prompt-induced weakly-supervised point cloud instance
segmentation (CIP-WPIS) method. CIP-WPIS leverages pretrained knowledge
embedded in the 2D foundation model SAM and 3D geometric prior to achieve
accurate point-wise instance labels from the bounding box annotations.
Specifically, CP-WPIS first selects image views in which 3D candidate points of
an instance are fully visible. Then, we generate complementary background and
foreground prompts from projections to obtain SAM 2D instance mask predictions.
According to these, we assign the confidence values to points indicating the
likelihood of points belonging to the instance. Furthermore, we utilize 3D
geometric homogeneity provided by superpoints to decide the final instance
label assignments. In this fashion, we achieve high-quality 3D point-wise
instance labels. Extensive experiments on both Scannet-v2 and S3DIS benchmarks
demonstrate that our method is robust against noisy 3D bounding-box annotations
and achieves state-of-the-art performance.