WarpDiffusion: Efficient Diffusion Model for High-Fidelity Virtual
  Try-on
xujie zhang Xiu Li Michael Kampffmeyer Xin Dong Zhenyu Xie Feida Zhu Haoye Dong Xiaodan Liang
Image-based Virtual Try-On (VITON) aims to transfer an in-shop garment image
onto a target person. While existing methods focus on warping the garment to
fit the body pose, they often overlook the synthesis quality around the
garment-skin boundary and realistic effects like wrinkles and shadows on the
warped garments. These limitations greatly reduce the realism of the generated
results and hinder the practical application of VITON techniques. Leveraging
the notable success of diffusion-based models in cross-modal image synthesis,
some recent diffusion-based methods have ventured to tackle this issue.
However, they tend to either consume a significant amount of training resources
or struggle to achieve realistic try-on effects and retain garment details. For
efficient and high-fidelity VITON, we propose WarpDiffusion, which bridges the
warping-based and diffusion-based paradigms via a novel informative and local
garment feature attention mechanism. Specifically, WarpDiffusion incorporates
local texture attention to reduce resource consumption and uses a novel
auto-mask module that effectively retains only the critical areas of the warped
garment while disregarding unrealistic or erroneous portions. Notably,
WarpDiffusion can be integrated as a plug-and-play component into existing
VITON methodologies, elevating their synthesis quality. Extensive experiments
on high-resolution VITON benchmarks and an in-the-wild test set demonstrate the
superiority of WarpDiffusion, surpassing state-of-the-art methods both
qualitatively and quantitatively.