Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes
Hmrishav Bandyopadhyay Subhadeep Koley Ayan Das Aneeshan Sain Pinaki Nath Chowdhury Tao Xiang Ayan Kumar Bhunia Yi-Zhe Song
In this paper, we democratise 3D content creation, enabling precise
generation of 3D shapes from abstract sketches while overcoming limitations
tied to drawing skills. We introduce a novel part-level modelling and alignment
framework that facilitates abstraction modelling and cross-modal
correspondence. Leveraging the same part-level decoder, our approach seamlessly
extends to sketch modelling by establishing correspondence between CLIPasso
edgemaps and projected 3D part regions, eliminating the need for a dataset
pairing human sketches and 3D shapes. Additionally, our method introduces a
seamless in-position editing process as a byproduct of cross-modal part-aligned
modelling. Operating in a low-dimensional implicit space, our approach
significantly reduces computational demands and processing time.