HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional
  Synthesis and Sampling of Hand-Object Interactions
Hao Xu Haipeng Li Yinqiao Wang Shuaicheng Liu Chi-Wing Fu
Reconstructing 3D hand mesh robustly from a single image is very challenging,
due to the lack of diversity in existing real-world datasets. While data
synthesis helps relieve the issue, the syn-to-real gap still hinders its usage.
In this work, we present HandBooster, a new approach to uplift the data
diversity and boost the 3D hand-mesh reconstruction performance by training a
conditional generative space on hand-object interactions and purposely sampling
the space to synthesize effective data samples. First, we construct versatile
content-aware conditions to guide a diffusion model to produce realistic images
with diverse hand appearances, poses, views, and backgrounds; favorably,
accurate 3D annotations are obtained for free. Then, we design a novel
condition creator based on our similarity-aware distribution sampling
strategies to deliberately find novel and realistic interaction poses that are
distinctive from the training set. Equipped with our method, several baselines
can be significantly improved beyond the SOTA on the HO3D and DexYCB
benchmarks. Our code will be released on
https://github.com/hxwork/HandBooster_Pytorch.