Weakly-Supervised 3D Reconstruction of Clothed Humans via Normal Maps
Jane Wu Diego Thomas Ronald Fedkiw
We present a novel deep learning-based approach to the 3D reconstruction of
clothed humans using weak supervision via 2D normal maps. Given a single RGB
image or multiview images, our network infers a signed distance function (SDF)
discretized on a tetrahedral mesh surrounding the body in a rest pose.
Subsequently, inferred pose and camera parameters are used to generate a normal
map from the SDF. A key aspect of our approach is the use of Marching
Tetrahedra to (uniquely) compute a triangulated surface from the SDF on the
tetrahedral mesh, facilitating straightforward differentiation (and thus
backpropagation). Thus, given only ground truth normal maps (with no volumetric
information ground truth information), we can train the network to produce SDF
values from corresponding RGB images. Optionally, an additional multiview loss
leads to improved results. We demonstrate the efficacy of our approach for both
network inference and 3D reconstruction.