Point Cloud Pre-training with Diffusion Models
Xiao Zheng Xiaoshui Huang Guofeng Mei Yuenan Hou Zhaoyang Lyu Bo Dai Wanli Ouyang Yongshun Gong
Pre-training a model and then fine-tuning it on downstream tasks has
demonstrated significant success in the 2D image and NLP domains. However, due
to the unordered and non-uniform density characteristics of point clouds, it is
non-trivial to explore the prior knowledge of point clouds and pre-train a
point cloud backbone. In this paper, we propose a novel pre-training method
called Point cloud Diffusion pre-training (PointDif). We consider the point
cloud pre-training task as a conditional point-to-point generation problem and
introduce a conditional point generator. This generator aggregates the features
extracted by the backbone and employs them as the condition to guide the
point-to-point recovery from the noisy point cloud, thereby assisting the
backbone in capturing both local and global geometric priors as well as the
global point density distribution of the object. We also present a recurrent
uniform sampling optimization strategy, which enables the model to uniformly
recover from various noise levels and learn from balanced supervision. Our
PointDif achieves substantial improvement across various real-world datasets
for diverse downstream tasks such as classification, segmentation and
detection. Specifically, PointDif attains 70.0% mIoU on S3DIS Area 5 for the
segmentation task and achieves an average improvement of 2.4% on ScanObjectNN
for the classification task compared to TAP. Furthermore, our pre-training
framework can be flexibly applied to diverse point cloud backbones and bring
considerable gains.