Temporal-Aware Refinement for Video-based Human Pose and Shape Recovery
Ming Chen Yan Zhou Weihua Jian Pengfei Wan Zhongyuan Wang
Though significant progress in human pose and shape recovery from monocular
RGB images has been made in recent years, obtaining 3D human motion with high
accuracy and temporal consistency from videos remains challenging. Existing
video-based methods tend to reconstruct human motion from global image
features, which lack detailed representation capability and limit the
reconstruction accuracy. In this paper, we propose a Temporal-Aware Refining
Network (TAR), to synchronously explore temporal-aware global and local image
features for accurate pose and shape recovery. First, a global transformer
encoder is introduced to obtain temporal global features from static feature
sequences. Second, a bidirectional ConvGRU network takes the sequence of
high-resolution feature maps as input, and outputs temporal local feature maps
that maintain high resolution and capture the local motion of the human body.
Finally, a recurrent refinement module iteratively updates estimated SMPL
parameters by leveraging both global and local temporal information to achieve
accurate and smooth results. Extensive experiments demonstrate that our TAR
obtains more accurate results than previous state-of-the-art methods on popular
benchmarks, i.e., 3DPW, MPI-INF-3DHP, and Human3.6M.