Annotated Hands for Generative Models
Yue Yang Atith N Gandhi Greg Turk
Generative models such as GANs and diffusion models have demonstrated
impressive image generation capabilities. Despite these successes, these
systems are surprisingly poor at creating images with hands. We propose a novel
training framework for generative models that substantially improves the
ability of such systems to create hand images. Our approach is to augment the
training images with three additional channels that provide annotations to
hands in the image. These annotations provide additional structure that coax
the generative model to produce higher quality hand images. We demonstrate this
approach on two different generative models: a generative adversarial network
and a diffusion model. We demonstrate our method both on a new synthetic
dataset of hand images and also on real photographs that contain hands. We
measure the improved quality of the generated hands through higher confidence
in finger joint identification using an off-the-shelf hand detector.