DiffHand: End-to-End Hand Mesh Reconstruction via Diffusion Models
Lijun Li Li'an Zhuo Bang Zhang Liefeng Bo Chen Chen
Hand mesh reconstruction from the monocular image is a challenging task due
to its depth ambiguity and severe occlusion, there remains a non-unique mapping
between the monocular image and hand mesh. To address this, we develop
DiffHand, the first diffusion-based framework that approaches hand mesh
reconstruction as a denoising diffusion process. Our one-stage pipeline
utilizes noise to model the uncertainty distribution of the intermediate hand
mesh in a forward process. We reformulate the denoising diffusion process to
gradually refine noisy hand mesh and then select mesh with the highest
probability of being correct based on the image itself, rather than relying on
2D joints extracted beforehand. To better model the connectivity of hand
vertices, we design a novel network module called the cross-modality decoder.
Extensive experiments on the popular benchmarks demonstrate that our method
outperforms the state-of-the-art hand mesh reconstruction approaches by
achieving 5.8mm PA-MPJPE on the Freihand test set, 4.98mm PA-MPJPE on the
DexYCB test set.