Detecting and Restoring Non-Standard Hands in Stable Diffusion Generated
  Images
Yiqun Zhang Zhenyue Qin Yang Liu Dylan Campbell
We introduce a pipeline to address anatomical inaccuracies in Stable
Diffusion generated hand images. The initial step involves constructing a
specialized dataset, focusing on hand anomalies, to train our models
effectively. A finetuned detection model is pivotal for precise identification
of these anomalies, ensuring targeted correction. Body pose estimation aids in
understanding hand orientation and positioning, crucial for accurate anomaly
correction. The integration of ControlNet and InstructPix2Pix facilitates
sophisticated inpainting and pixel-level transformation, respectively. This
dual approach allows for high-fidelity image adjustments. This comprehensive
approach ensures the generation of images with anatomically accurate hands,
closely resembling real-world appearances. Our experimental results demonstrate
the pipeline's efficacy in enhancing hand image realism in Stable Diffusion
outputs. We provide an online demo at https://fixhand.yiqun.io