Diffusion Shape Prior for Wrinkle-Accurate Cloth Registration
Jingfan Guo Fabian Prada Donglai Xiang Javier Romero Chenglei Wu Hyun Soo Park Takaaki Shiratori Shunsuke Saito
Registering clothes from 4D scans with vertex-accurate correspondence is
challenging, yet important for dynamic appearance modeling and physics
parameter estimation from real-world data. However, previous methods either
rely on texture information, which is not always reliable, or achieve only
coarse-level alignment. In this work, we present a novel approach to enabling
accurate surface registration of texture-less clothes with large deformation.
Our key idea is to effectively leverage a shape prior learned from pre-captured
clothing using diffusion models. We also propose a multi-stage guidance scheme
based on learned functional maps, which stabilizes registration for large-scale
deformation even when they vary significantly from training data. Using
high-fidelity real captured clothes, our experiments show that the proposed
approach based on diffusion models generalizes better than surface registration
with VAE or PCA-based priors, outperforming both optimization-based and
learning-based non-rigid registration methods for both interpolation and
extrapolation tests.