TPA3D: Triplane Attention for Fast Text-to-3D Generation
Hong-En Chen Bin-Shih Wu Sheng-Yu Huang Yu-Chiang Frank Wang
Due to the lack of large-scale text-3D correspondence data, recent text-to-3D
generation works mainly rely on utilizing 2D diffusion models for synthesizing
3D data. Since diffusion-based methods typically require significant
optimization time for both training and inference, the use of GAN-based models
would still be desirable for fast 3D generation. In this work, we propose
Triplane Attention for text-guided 3D generation (TPA3D), an end-to-end
trainable GAN-based deep learning model for fast text-to-3D generation. With
only 3D shape data and their rendered 2D images observed during training, our
TPA3D is designed to retrieve detailed visual descriptions for synthesizing the
corresponding 3D mesh data. This is achieved by the proposed attention
mechanisms on the extracted sentence and word-level text features. In our
experiments, we show that TPA3D generates high-quality 3D textured shapes
aligned with fine-grained descriptions, while impressive computation efficiency
can be observed.