GET3D--: Learning GET3D from Unconstrained Image Collections
Fanghua Yu Xintao Wang Zheyuan Li Yan-Pei Cao Ying Shan Chao Dong
The demand for efficient 3D model generation techniques has grown
exponentially, as manual creation of 3D models is time-consuming and requires
specialized expertise. While generative models have shown potential in creating
3D textured shapes from 2D images, their applicability in 3D industries is
limited due to the lack of a well-defined camera distribution in real-world
scenarios, resulting in low-quality shapes. To overcome this limitation, we
propose GET3D--, the first method that directly generates textured 3D shapes
from 2D images with unknown pose and scale. GET3D-- comprises a 3D shape
generator and a learnable camera sampler that captures the 6D external changes
on the camera. In addition, We propose a novel training schedule to stably
optimize both the shape generator and camera sampler in a unified framework. By
controlling external variations using the learnable camera sampler, our method
can generate aligned shapes with clear textures. Extensive experiments
demonstrate the efficacy of GET3D--, which precisely fits the 6D camera pose
distribution and generates high-quality shapes on both synthetic and realistic
unconstrained datasets.