3Deformer: A Common Framework for Image-Guided Mesh Deformation
Hao Su Xuefeng Liu Jianwei Niu Ji Wan Xinghao Wu
We propose 3Deformer, a general-purpose framework for interactive 3D shape
editing. Given a source 3D mesh with semantic materials, and a user-specified
semantic image, 3Deformer can accurately edit the source mesh following the
shape guidance of the semantic image, while preserving the source topology as
rigid as possible. Recent studies of 3D shape editing mostly focus on learning
neural networks to predict 3D shapes, which requires high-cost 3D training
datasets and is limited to handling objects involved in the datasets. Unlike
these studies, our 3Deformer is a non-training and common framework, which only
requires supervision of readily-available semantic images, and is compatible
with editing various objects unlimited by datasets. In 3Deformer, the source
mesh is deformed utilizing the differentiable renderer technique, according to
the correspondences between semantic images and mesh materials. However,
guiding complex 3D shapes with a simple 2D image incurs extra challenges, that
is, the deform accuracy, surface smoothness, geometric rigidity, and global
synchronization of the edited mesh should be guaranteed. To address these
challenges, we propose a hierarchical optimization architecture to balance the
global and local shape features, and propose further various strategies and
losses to improve properties of accuracy, smoothness, rigidity, and so on.
Extensive experiments show that our 3Deformer is able to produce impressive
results and reaches the state-of-the-art level.