General Point Model with Autoencoding and Autoregressive
Zhe Li Zhangyang Gao Cheng Tan Stan Z. Li Laurence T. Yang
The pre-training architectures of large language models encompass various
types, including autoencoding models, autoregressive models, and
encoder-decoder models. We posit that any modality can potentially benefit from
a large language model, as long as it undergoes vector quantization to become
discrete tokens. Inspired by GLM, we propose a General Point Model (GPM) which
seamlessly integrates autoencoding and autoregressive tasks in point cloud
transformer. This model is versatile, allowing fine-tuning for downstream point
cloud representation tasks, as well as unconditional and conditional generation
tasks. GPM enhances masked prediction in autoencoding through various forms of
mask padding tasks, leading to improved performance in point cloud
understanding. Additionally, GPM demonstrates highly competitive results in
unconditional point cloud generation tasks, even exhibiting the potential for
conditional generation tasks by modifying the input's conditional information.
Compared to models like Point-BERT, MaskPoint and PointMAE, our GPM achieves
superior performance in point cloud understanding tasks. Furthermore, the
integration of autoregressive and autoencoding within the same transformer
underscores its versatility across different downstream tasks.