ET3D: Efficient Text-to-3D Generation via Multi-View Distillation
Yiming Chen Zhiqi Li Peidong Liu
Recent breakthroughs in text-to-image generation has shown encouraging
results via large generative models. Due to the scarcity of 3D assets, it is
hardly to transfer the success of text-to-image generation to that of
text-to-3D generation. Existing text-to-3D generation methods usually adopt the
paradigm of DreamFusion, which conducts per-asset optimization by distilling a
pretrained text-to-image diffusion model. The generation speed usually ranges
from several minutes to tens of minutes per 3D asset, which degrades the user
experience and also imposes a burden to the service providers due to the high
computational budget.
  In this work, we present an efficient text-to-3D generation method, which
requires only around 8 $ms$ to generate a 3D asset given the text prompt on a
consumer graphic card. The main insight is that we exploit the images generated
by a large pre-trained text-to-image diffusion model, to supervise the training
of a text conditioned 3D generative adversarial network. Once the network is
trained, we are able to efficiently generate a 3D asset via a single forward
pass. Our method requires no 3D training data and provides an alternative
approach for efficient text-to-3D generation by distilling pre-trained image
diffusion models.