3D Hand Pose Estimation in Egocentric Images in the Wild
Aditya Prakash Ruisen Tu Matthew Chang Saurabh Gupta
We present WildHands, a method for 3D hand pose estimation in egocentric
images in the wild. This is challenging due to (a) lack of 3D hand pose
annotations for images in the wild, and (b) a form of perspective
distortion-induced shape ambiguity that arises in the analysis of crops around
hands. For the former, we use auxiliary supervision on in-the-wild data in the
form of segmentation masks & grasp labels in addition to 3D supervision
available in lab datasets. For the latter, we provide spatial cues about the
location of the hand crop in the camera's field of view. Our approach achieves
the best 3D hand pose on the ARCTIC leaderboard and outperforms FrankMocap, a
popular and robust approach for estimating hand pose in the wild, by 45.3% when
evaluated on 2D hand pose on our EPIC-HandKps dataset.