CAT-DM: Controllable Accelerated Virtual Try-on with Diffusion Model
Jianhao Zeng Dan Song Weizhi Nie Hongshuo Tian Tongtong Wang Anan Liu
Image-based virtual try-on enables users to virtually try on different
garments by altering original clothes in their photographs. Generative
Adversarial Networks (GANs) dominate the research field in image-based virtual
try-on, but have not resolved problems such as unnatural deformation of
garments and the blurry generation quality. Recently, diffusion models have
emerged with surprising performance across various image generation tasks.
While the generative quality of diffusion models is impressive, achieving
controllability poses a significant challenge when applying it to virtual
try-on tasks and multiple denoising iterations limit its potential for
real-time applications. In this paper, we propose Controllable Accelerated
virtual Try-on with Diffusion Model called CAT-DM. To enhance the
controllability, a basic diffusion-based virtual try-on network is designed,
which utilizes ControlNet to introduce additional control conditions and
improves the feature extraction of garment images. In terms of acceleration,
CAT-DM initiates a reverse denoising process with an implicit distribution
generated by a pre-trained GAN-based model. Compared with previous try-on
methods based on diffusion models, CAT-DM not only retains the pattern and
texture details of the in-shop garment but also reduces the sampling steps
without compromising generation quality. Extensive experiments demonstrate the
superiority of CAT-DM against both GAN-based and diffusion-based methods in
producing more realistic images and accurately reproducing garment patterns.
Our code and models will be publicly released.