SHaRPose: Sparse High-Resolution Representation for Human Pose
  Estimation
Xiaoqi An Lin Zhao Chen Gong Nannan Wang Di Wang Jian Yang
High-resolution representation is essential for achieving good performance in
human pose estimation models. To obtain such features, existing works utilize
high-resolution input images or fine-grained image tokens. However, this dense
high-resolution representation brings a significant computational burden. In
this paper, we address the following question: "Only sparse human keypoint
locations are detected for human pose estimation, is it really necessary to
describe the whole image in a dense, high-resolution manner?" Based on dynamic
transformer models, we propose a framework that only uses Sparse
High-resolution Representations for human Pose estimation (SHaRPose). In
detail, SHaRPose consists of two stages. At the coarse stage, the relations
between image regions and keypoints are dynamically mined while a coarse
estimation is generated. Then, a quality predictor is applied to decide whether
the coarse estimation results should be refined. At the fine stage, SHaRPose
builds sparse high-resolution representations only on the regions related to
the keypoints and provides refined high-precision human pose estimations.
Extensive experiments demonstrate the outstanding performance of the proposed
method. Specifically, compared to the state-of-the-art method ViTPose, our
model SHaRPose-Base achieves 77.4 AP (+0.5 AP) on the COCO validation set and
76.7 AP (+0.5 AP) on the COCO test-dev set, and infers at a speed of
$1.4\times$ faster than ViTPose-Base.