On the Calibration of Human Pose Estimation
Kerui Gu Rongyu Chen Angela Yao
Most 2D human pose estimation frameworks estimate keypoint confidence in an
ad-hoc manner, using heuristics such as the maximum value of heatmaps. The
confidence is part of the evaluation scheme, e.g., AP for the MSCOCO dataset,
yet has been largely overlooked in the development of state-of-the-art methods.
This paper takes the first steps in addressing miscalibration in pose
estimation. From a calibration point of view, the confidence should be aligned
with the pose accuracy. In practice, existing methods are poorly calibrated. We
show, through theoretical analysis, why a miscalibration gap exists and how to
narrow the gap. Simply predicting the instance size and adjusting the
confidence function gives considerable AP improvements. Given the black-box
nature of deep neural networks, however, it is not possible to fully close this
gap with only closed-form adjustments. As such, we go one step further and
learn network-specific adjustments by enforcing consistency between confidence
and pose accuracy. Our proposed Calibrated ConfidenceNet (CCNet) is a
light-weight post-hoc addition that improves AP by up to 1.4% on off-the-shelf
pose estimation frameworks. Applied to the downstream task of mesh recovery,
CCNet facilitates an additional 1.0mm decrease in 3D keypoint error.