Deep3DSketch+: Rapid 3D Modeling from Single Free-hand Sketches
Tianrun Chen Chenglong Fu Ying Zang Lanyun Zhu Jia Zhang Papa Mao Lingyun Sun
The rapid development of AR/VR brings tremendous demands for 3D content.
While the widely-used Computer-Aided Design (CAD) method requires a
time-consuming and labor-intensive modeling process, sketch-based 3D modeling
offers a potential solution as a natural form of computer-human interaction.
However, the sparsity and ambiguity of sketches make it challenging to generate
high-fidelity content reflecting creators' ideas. Precise drawing from multiple
views or strategic step-by-step drawings is often required to tackle the
challenge but is not friendly to novice users. In this work, we introduce a
novel end-to-end approach, Deep3DSketch+, which performs 3D modeling using only
a single free-hand sketch without inputting multiple sketches or view
information. Specifically, we introduce a lightweight generation network for
efficient inference in real-time and a structural-aware adversarial training
approach with a Stroke Enhancement Module (SEM) to capture the structural
information to facilitate learning of the realistic and fine-detailed shape
structures for high-fidelity performance. Extensive experiments demonstrated
the effectiveness of our approach with the state-of-the-art (SOTA) performance
on both synthetic and real datasets.