Neural Interactive Keypoint Detection
Jie Yang Ailing Zeng Feng Li Shilong Liu Ruimao Zhang Lei Zhang
This work proposes an end-to-end neural interactive keypoint detection
framework named Click-Pose, which can significantly reduce more than 10 times
labeling costs of 2D keypoint annotation compared with manual-only annotation.
Click-Pose explores how user feedback can cooperate with a neural keypoint
detector to correct the predicted keypoints in an interactive way for a faster
and more effective annotation process. Specifically, we design the pose error
modeling strategy that inputs the ground truth pose combined with four typical
pose errors into the decoder and trains the model to reconstruct the correct
poses, which enhances the self-correction ability of the model. Then, we attach
an interactive human-feedback loop that allows receiving users' clicks to
correct one or several predicted keypoints and iteratively utilizes the decoder
to update all other keypoints with a minimum number of clicks (NoC) for
efficient annotation. We validate Click-Pose in in-domain, out-of-domain
scenes, and a new task of keypoint adaptation. For annotation, Click-Pose only
needs 1.97 and 6.45 NoC@95 (at precision 95%) on COCO and Human-Art, reducing
31.4% and 36.3% efforts than the SOTA model (ViTPose) with manual correction,
respectively. Besides, without user clicks, Click-Pose surpasses the previous
end-to-end model by 1.4 AP on COCO and 3.0 AP on Human-Art. The code is
available at https://github.com/IDEA-Research/Click-Pose.