Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On
Xu Yang Changxing Ding Zhibin Hong Junhao Huang Jin Tao Xiangmin Xu
Image-based virtual try-on is an increasingly important task for online
shopping. It aims to synthesize images of a specific person wearing a specified
garment. Diffusion model-based approaches have recently become popular, as they
are excellent at image synthesis tasks. However, these approaches usually
employ additional image encoders and rely on the cross-attention mechanism for
texture transfer from the garment to the person image, which affects the
try-on's efficiency and fidelity. To address these issues, we propose an
Texture-Preserving Diffusion (TPD) model for virtual try-on, which enhances the
fidelity of the results and introduces no additional image encoders.
Accordingly, we make contributions from two aspects. First, we propose to
concatenate the masked person and reference garment images along the spatial
dimension and utilize the resulting image as the input for the diffusion
model's denoising UNet. This enables the original self-attention layers
contained in the diffusion model to achieve efficient and accurate texture
transfer. Second, we propose a novel diffusion-based method that predicts a
precise inpainting mask based on the person and reference garment images,
further enhancing the reliability of the try-on results. In addition, we
integrate mask prediction and image synthesis into a single compact model. The
experimental results show that our approach can be applied to various try-on
tasks, e.g., garment-to-person and person-to-person try-ons, and significantly
outperforms state-of-the-art methods on popular VITON, VITON-HD databases.