Lifting by Image -- Leveraging Image Cues for Accurate 3D Human Pose
  Estimation
Feng Zhou Jianqin Yin Peiyang Li
The "lifting from 2D pose" method has been the dominant approach to 3D Human
Pose Estimation (3DHPE) due to the powerful visual analysis ability of 2D pose
estimators. Widely known, there exists a depth ambiguity problem when
estimating solely from 2D pose, where one 2D pose can be mapped to multiple 3D
poses. Intuitively, the rich semantic and texture information in images can
contribute to a more accurate "lifting" procedure. Yet, existing research
encounters two primary challenges. Firstly, the distribution of image data in
3D motion capture datasets is too narrow because of the laboratorial
environment, which leads to poor generalization ability of methods trained with
image information. Secondly, effective strategies for leveraging image
information are lacking. In this paper, we give new insight into the cause of
poor generalization problems and the effectiveness of image features. Based on
that, we propose an advanced framework. Specifically, the framework consists of
two stages. First, we enable the keypoints to query and select the beneficial
features from all image patches. To reduce the keypoints attention to
inconsequential background features, we design a novel Pose-guided Transformer
Layer, which adaptively limits the updates to unimportant image patches. Then,
through a designed Adaptive Feature Selection Module, we prune less significant
image patches from the feature map. In the second stage, we allow the keypoints
to further emphasize the retained critical image features. This progressive
learning approach prevents further training on insignificant image features.
Experimental results show that our model achieves state-of-the-art performance
on both the Human3.6M dataset and the MPI-INF-3DHP dataset.