Prompt2NeRF-PIL: Fast NeRF Generation via Pretrained Implicit Latent
Jianmeng Liu Yuyao Zhang Zeyuan Meng Yu-Wing Tai Chi-Keung Tang
This paper explores promptable NeRF generation (e.g., text prompt or single
image prompt) for direct conditioning and fast generation of NeRF parameters
for the underlying 3D scenes, thus undoing complex intermediate steps while
providing full 3D generation with conditional control. Unlike previous
diffusion-CLIP-based pipelines that involve tedious per-prompt optimizations,
Prompt2NeRF-PIL is capable of generating a variety of 3D objects with a single
forward pass, leveraging a pre-trained implicit latent space of NeRF
parameters. Furthermore, in zero-shot tasks, our experiments demonstrate that
the NeRFs produced by our method serve as semantically informative
initializations, significantly accelerating the inference process of existing
prompt-to-NeRF methods. Specifically, we will show that our approach speeds up
the text-to-NeRF model DreamFusion and the 3D reconstruction speed of the
image-to-NeRF method Zero-1-to-3 by 3 to 5 times.