Learning Human Mesh Recovery in 3D Scenes
Zehong Shen Zhi Cen Sida Peng Qing Shuai Hujun Bao Xiaowei Zhou
We present a novel method for recovering the absolute pose and shape of a
human in a pre-scanned scene given a single image. Unlike previous methods that
perform sceneaware mesh optimization, we propose to first estimate absolute
position and dense scene contacts with a sparse 3D CNN, and later enhance a
pretrained human mesh recovery network by cross-attention with the derived 3D
scene cues. Joint learning on images and scene geometry enables our method to
reduce the ambiguity caused by depth and occlusion, resulting in more
reasonable global postures and contacts. Encoding scene-aware cues in the
network also allows the proposed method to be optimization-free, and opens up
the opportunity for real-time applications. The experiments show that the
proposed network is capable of recovering accurate and physically-plausible
meshes by a single forward pass and outperforms state-of-the-art methods in
terms of both accuracy and speed.