Lightweight Super-Resolution Head for Human Pose Estimation
Haonan Wang Jie Liu Jie Tang Gangshan Wu
Heatmap-based methods have become the mainstream method for pose estimation
due to their superior performance. However, heatmap-based approaches suffer
from significant quantization errors with downscale heatmaps, which result in
limited performance and the detrimental effects of intermediate supervision.
Previous heatmap-based methods relied heavily on additional post-processing to
mitigate quantization errors. Some heatmap-based approaches improve the
resolution of feature maps by using multiple costly upsampling layers to
improve localization precision. To solve the above issues, we creatively view
the backbone network as a degradation process and thus reformulate the heatmap
prediction as a Super-Resolution (SR) task. We first propose the SR head, which
predicts heatmaps with a spatial resolution higher than the input feature maps
(or even consistent with the input image) by super-resolution, to effectively
reduce the quantization error and the dependence on further post-processing.
Besides, we propose SRPose to gradually recover the HR heatmaps from LR
heatmaps and degraded features in a coarse-to-fine manner. To reduce the
training difficulty of HR heatmaps, SRPose applies SR heads to supervise the
intermediate features in each stage. In addition, the SR head is a lightweight
and generic head that applies to top-down and bottom-up methods. Extensive
experiments on the COCO, MPII, and CrowdPose datasets show that SRPose
outperforms the corresponding heatmap-based approaches. The code and models are
available at https://github.com/haonanwang0522/SRPose.