P2M2-Net: Part-Aware Prompt-Guided Multimodal Point Cloud Completion
Linlian Jiang Pan Chen Ye Wang Tieru Wu Rui Ma
Inferring missing regions from severely occluded point clouds is highly
challenging. Especially for 3D shapes with rich geometry and structure details,
inherent ambiguities of the unknown parts are existing. Existing approaches
either learn a one-to-one mapping in a supervised manner or train a generative
model to synthesize the missing points for the completion of 3D point cloud
shapes. These methods, however, lack the controllability for the completion
process and the results are either deterministic or exhibiting uncontrolled
diversity. Inspired by the prompt-driven data generation and editing, we
propose a novel prompt-guided point cloud completion framework, coined
P2M2-Net, to enable more controllable and more diverse shape completion. Given
an input partial point cloud and a text prompt describing the part-aware
information such as semantics and structure of the missing region, our
Transformer-based completion network can efficiently fuse the multimodal
features and generate diverse results following the prompt guidance. We train
the P2M2-Net on a new large-scale PartNet-Prompt dataset and conduct extensive
experiments on two challenging shape completion benchmarks. Quantitative and
qualitative results show the efficacy of incorporating prompts for more
controllable part-aware point cloud completion and generation. Code and data
are available at https://github.com/JLU-ICL/P2M2-Net.