Self-supervised Pre-training with Masked Shape Prediction for 3D Scene
  Understanding
Li Jiang Zetong Yang Shaoshuai Shi Vladislav Golyanik Dengxin Dai Bernt Schiele
Masked signal modeling has greatly advanced self-supervised pre-training for
language and 2D images. However, it is still not fully explored in 3D scene
understanding. Thus, this paper introduces Masked Shape Prediction (MSP), a new
framework to conduct masked signal modeling in 3D scenes. MSP uses the
essential 3D semantic cue, i.e., geometric shape, as the prediction target for
masked points. The context-enhanced shape target consisting of explicit shape
context and implicit deep shape feature is proposed to facilitate exploiting
contextual cues in shape prediction. Meanwhile, the pre-training architecture
in MSP is carefully designed to alleviate the masked shape leakage from point
coordinates. Experiments on multiple 3D understanding tasks on both indoor and
outdoor datasets demonstrate the effectiveness of MSP in learning good feature
representations to consistently boost downstream performance.