MoCap-to-Visual Domain Adaptation for Efficient Human Mesh Estimation
  from 2D Keypoints
Bedirhan Uguz Ozhan Suat Batuhan Karagoz Emre Akbas
This paper presents Key2Mesh, a model that takes a set of 2D human pose
keypoints as input and estimates the corresponding body mesh. Since this
process does not involve any visual (i.e. RGB image) data, the model can be
trained on large-scale motion capture (MoCap) datasets, thereby overcoming the
scarcity of image datasets with 3D labels. To enable the model's application on
RGB images, we first run an off-the-shelf 2D pose estimator to obtain the 2D
keypoints, and then feed these 2D keypoints to Key2Mesh. To improve the
performance of our model on RGB images, we apply an adversarial domain
adaptation (DA) method to bridge the gap between the MoCap and visual domains.
Crucially, our DA method does not require 3D labels for visual data, which
enables adaptation to target sets without the need for costly labels. We
evaluate Key2Mesh for the task of estimating 3D human meshes from 2D keypoints,
in the absence of RGB and mesh label pairs. Our results on widely used H3.6M
and 3DPW datasets show that Key2Mesh sets the new state-of-the-art by
outperforming other models in PA-MPJPE for both datasets, and in MPJPE and PVE
for the 3DPW dataset. Thanks to our model's simple architecture, it operates at
least 12x faster than the prior state-of-the-art model, LGD. Additional
qualitative samples and code are available on the project website:
https://key2mesh.github.io/.