Rethinking Few-shot 3D Point Cloud Semantic Segmentation
Zhaochong An Guolei Sun Yun Liu Fayao Liu Zongwei Wu Dan Wang Luc Van Gool Serge Belongie
This paper revisits few-shot 3D point cloud semantic segmentation (FS-PCS),
with a focus on two significant issues in the state-of-the-art: foreground
leakage and sparse point distribution. The former arises from non-uniform point
sampling, allowing models to distinguish the density disparities between
foreground and background for easier segmentation. The latter results from
sampling only 2,048 points, limiting semantic information and deviating from
the real-world practice. To address these issues, we introduce a standardized
FS-PCS setting, upon which a new benchmark is built. Moreover, we propose a
novel FS-PCS model. While previous methods are based on feature optimization by
mainly refining support features to enhance prototypes, our method is based on
correlation optimization, referred to as Correlation Optimization Segmentation
(COSeg). Specifically, we compute Class-specific Multi-prototypical Correlation
(CMC) for each query point, representing its correlations to category
prototypes. Then, we propose the Hyper Correlation Augmentation (HCA) module to
enhance CMC. Furthermore, tackling the inherent property of few-shot training
to incur base susceptibility for models, we propose to learn non-parametric
prototypes for the base classes during training. The learned base prototypes
are used to calibrate correlations for the background class through a Base
Prototypes Calibration (BPC) module. Experiments on popular datasets
demonstrate the superiority of COSeg over existing methods. The code is
available at: https://github.com/ZhaochongAn/COSeg