Point Contrastive Prediction with Semantic Clustering for
  Self-Supervised Learning on Point Cloud Videos
Xiaoxiao Sheng Zhiqiang Shen Gang Xiao Longguang Wang Yulan Guo Hehe Fan
We propose a unified point cloud video self-supervised learning framework for
object-centric and scene-centric data. Previous methods commonly conduct
representation learning at the clip or frame level and cannot well capture
fine-grained semantics. Instead of contrasting the representations of clips or
frames, in this paper, we propose a unified self-supervised framework by
conducting contrastive learning at the point level. Moreover, we introduce a
new pretext task by achieving semantic alignment of superpoints, which further
facilitates the representations to capture semantic cues at multiple scales. In
addition, due to the high redundancy in the temporal dimension of dynamic point
clouds, directly conducting contrastive learning at the point level usually
leads to massive undesired negatives and insufficient modeling of positive
representations. To remedy this, we propose a selection strategy to retain
proper negatives and make use of high-similarity samples from other instances
as positive supplements. Extensive experiments show that our method outperforms
supervised counterparts on a wide range of downstream tasks and demonstrates
the superior transferability of the learned representations.