3D Hand Reconstruction via Aggregating Intra and Inter Graphs Guided by
  Prior Knowledge for Hand-Object Interaction Scenario
Feng Shuang Wenbo He Shaodong Li
Recently, 3D hand reconstruction has gained more attention in human-computer
cooperation, especially for hand-object interaction scenario. However, it still
remains huge challenge due to severe hand-occlusion caused by interaction,
which contain the balance of accuracy and physical plausibility, highly
nonlinear mapping of model parameters and occlusion feature enhancement. To
overcome these issues, we propose a 3D hand reconstruction network combining
the benefits of model-based and model-free approaches to balance accuracy and
physical plausibility for hand-object interaction scenario. Firstly, we present
a novel MANO pose parameters regression module from 2D joints directly, which
avoids the process of highly nonlinear mapping from abstract image feature and
no longer depends on accurate 3D joints. Moreover, we further propose a
vertex-joint mutual graph-attention model guided by MANO to jointly refine hand
meshes and joints, which model the dependencies of vertex-vertex and
joint-joint and capture the correlation of vertex-joint for aggregating
intra-graph and inter-graph node features respectively. The experimental
results demonstrate that our method achieves a competitive performance on
recently benchmark datasets HO3DV2 and Dex-YCB, and outperforms all only
model-base approaches and model-free approaches.