DCS-Net: Pioneering Leakage-Free Point Cloud Pretraining Framework with
  Global Insights
Zhe Li Zhangyang Gao Cheng Tan Stan Z. Li Laurence T. Yang
Masked autoencoding and generative pretraining have achieved remarkable
success in computer vision and natural language processing, and more recently,
they have been extended to the point cloud domain. Nevertheless, existing point
cloud models suffer from the issue of information leakage due to the
pre-sampling of center points, which leads to trivial proxy tasks for the
models. These approaches primarily focus on local feature reconstruction,
limiting their ability to capture global patterns within point clouds. In this
paper, we argue that the reduced difficulty of pretext tasks hampers the
model's capacity to learn expressive representations. To address these
limitations, we introduce a novel solution called the Differentiable Center
Sampling Network (DCS-Net). It tackles the information leakage problem by
incorporating both global feature reconstruction and local feature
reconstruction as non-trivial proxy tasks, enabling simultaneous learning of
both the global and local patterns within point cloud. Experimental results
demonstrate that our method enhances the expressive capacity of existing point
cloud models and effectively addresses the issue of information leakage.