RSB-Pose: Robust Short-Baseline Binocular 3D Human Pose Estimation with
  Occlusion Handling
Xiaoyue Wan Zhuo Chen Yiming Bao Xu Zhao
In the domain of 3D Human Pose Estimation, which finds widespread daily
applications, the requirement for convenient acquisition equipment continues to
grow. To satisfy this demand, we set our sights on a short-baseline binocular
setting that offers both portability and a geometric measurement property that
radically mitigates depth ambiguity. However, as the binocular baseline
shortens, two serious challenges emerge: first, the robustness of 3D
reconstruction against 2D errors deteriorates; and second, occlusion reoccurs
due to the limited visual differences between two views. To address the first
challenge, we propose the Stereo Co-Keypoints Estimation module to improve the
view consistency of 2D keypoints and enhance the 3D robustness. In this module,
the disparity is utilized to represent the correspondence of binocular 2D
points and the Stereo Volume Feature is introduced to contain binocular
features across different disparities. Through the regression of SVF, two-view
2D keypoints are simultaneously estimated in a collaborative way which
restricts their view consistency. Furthermore, to deal with occlusions, a
Pre-trained Pose Transformer module is introduced. Through this module, 3D
poses are refined by perceiving pose coherence, a representation of joint
correlations. This perception is injected by the Pose Transformer network and
learned through a pre-training task that recovers iterative masked joints.
Comprehensive experiments carried out on H36M and MHAD datasets, complemented
by visualizations, validate the effectiveness of our approach in the
short-baseline binocular 3D Human Pose Estimation and occlusion handling.