A Two-stage Personalized Virtual Try-on Framework with Shape Control and
  Texture Guidance
Shufang Zhang Minxue Ni Lei Wang Wenxin Ding Shuai Chen Yuhong Liu
The Diffusion model has a strong ability to generate wild images. However,
the model can just generate inaccurate images with the guidance of text, which
makes it very challenging to directly apply the text-guided generative model
for virtual try-on scenarios. Taking images as guiding conditions of the
diffusion model, this paper proposes a brand new personalized virtual try-on
model (PE-VITON), which uses the two stages (shape control and texture
guidance) to decouple the clothing attributes. Specifically, the proposed model
adaptively matches the clothing to human body parts through the Shape Control
Module (SCM) to mitigate the misalignment of the clothing and the human body
parts. The semantic information of the input clothing is parsed by the Texture
Guided Module (TGM), and the corresponding texture is generated by directional
guidance. Therefore, this model can effectively solve the problems of weak
reduction of clothing folds, poor generation effect under complex human
posture, blurred edges of clothing, and unclear texture styles in traditional
try-on methods. Meanwhile, the model can automatically enhance the generated
clothing folds and textures according to the human posture, and improve the
authenticity of virtual try-on. In this paper, qualitative and quantitative
experiments are carried out on high-resolution paired and unpaired datasets,
the results show that the proposed model outperforms the state-of-the-art
model.