Textureless Deformable Surface Reconstruction with Invisible Markers
Xinyuan Li Yu Ji Yanchen Liu Xiaochen Hu Jinwei Ye Changxi Zheng
Reconstructing and tracking deformable surface with little or no texture has
posed long-standing challenges. Fundamentally, the challenges stem from
textureless surfaces lacking features for establishing cross-image
correspondences. In this work, we present a novel type of markers to
proactively enrich the object's surface features, and thereby ease the 3D
surface reconstruction and correspondence tracking. Our markers are made of
fluorescent dyes, visible only under the ultraviolet (UV) light and invisible
under regular lighting condition. Leveraging the markers, we design a
multi-camera system that captures surface deformation under the UV light and
the visible light in a time multiplexing fashion. Under the UV light, markers
on the object emerge to enrich its surface texture, allowing high-quality 3D
shape reconstruction and tracking. Under the visible light, markers become
invisible, allowing us to capture the object's original untouched appearance.
We perform experiments on various challenging scenes, including hand gestures,
facial expressions, waving cloth, and hand-object interaction. In all these
cases, we demonstrate that our system is able to produce robust, high-quality
3D reconstruction and tracking.