HandGCAT: Occlusion-Robust 3D Hand Mesh Reconstruction from Monocular
  Images
Shuaibing Wang Shunli Wang Dingkang Yang Mingcheng Li Ziyun Qian Liuzhen Su Lihua Zhang
We propose a robust and accurate method for reconstructing 3D hand mesh from
monocular images. This is a very challenging problem, as hands are often
severely occluded by objects. Previous works often have disregarded 2D hand
pose information, which contains hand prior knowledge that is strongly
correlated with occluded regions. Thus, in this work, we propose a novel 3D
hand mesh reconstruction network HandGCAT, that can fully exploit hand prior as
compensation information to enhance occluded region features. Specifically, we
designed the Knowledge-Guided Graph Convolution (KGC) module and the
Cross-Attention Transformer (CAT) module. KGC extracts hand prior information
from 2D hand pose by graph convolution. CAT fuses hand prior into occluded
regions by considering their high correlation. Extensive experiments on popular
datasets with challenging hand-object occlusions, such as HO3D v2, HO3D v3, and
DexYCB demonstrate that our HandGCAT reaches state-of-the-art performance. The
code is available at https://github.com/heartStrive/HandGCAT.