M$^3$CS: Multi-Target Masked Point Modeling with Learnable Codebook and
  Siamese Decoders
Qibo Qiu Honghui Yang Wenxiao Wang Shun Zhang Haiming Gao Haochao Ying Wei Hua Xiaofei He
Masked point modeling has become a promising scheme of self-supervised
pre-training for point clouds. Existing methods reconstruct either the original
points or related features as the objective of pre-training. However,
considering the diversity of downstream tasks, it is necessary for the model to
have both low- and high-level representation modeling capabilities to capture
geometric details and semantic contexts during pre-training. To this end,
M$^3$CS is proposed to enable the model with the above abilities. Specifically,
with masked point cloud as input, M$^3$CS introduces two decoders to predict
masked representations and the original points simultaneously. While an extra
decoder doubles parameters for the decoding process and may lead to
overfitting, we propose siamese decoders to keep the amount of learnable
parameters unchanged. Further, we propose an online codebook projecting
continuous tokens into discrete ones before reconstructing masked points. In
such way, we can enforce the decoder to take effect through the combinations of
tokens rather than remembering each token. Comprehensive experiments show that
M$^3$CS achieves superior performance at both classification and segmentation
tasks, outperforming existing methods.