WordRobe: Text-Guided Generation of Textured 3D Garments
Astitva Srivastava Pranav Manu Amit Raj Varun Jampani Avinash Sharma
In this paper, we tackle a new and challenging problem of text-driven
generation of 3D garments with high-quality textures. We propose "WordRobe", a
novel framework for the generation of unposed & textured 3D garment meshes from
user-friendly text prompts. We achieve this by first learning a latent
representation of 3D garments using a novel coarse-to-fine training strategy
and a loss for latent disentanglement, promoting better latent interpolation.
Subsequently, we align the garment latent space to the CLIP embedding space in
a weakly supervised manner, enabling text-driven 3D garment generation and
editing. For appearance modeling, we leverage the zero-shot generation
capability of ControlNet to synthesize view-consistent texture maps in a single
feed-forward inference step, thereby drastically decreasing the generation time
as compared to existing methods. We demonstrate superior performance over
current SOTAs for learning 3D garment latent space, garment interpolation, and
text-driven texture synthesis, supported by quantitative evaluation and
qualitative user study. The unposed 3D garment meshes generated using WordRobe
can be directly fed to standard cloth simulation & animation pipelines without
any post-processing.