StableGarment: Garment-Centric Generation via Stable Diffusion
Rui Wang Hailong Guo Jiaming Liu Huaxia Li Haibo Zhao Xu Tang Yao Hu Hao Tang Peipei Li
In this paper, we introduce StableGarment, a unified framework to tackle
garment-centric(GC) generation tasks, including GC text-to-image, controllable
GC text-to-image, stylized GC text-to-image, and robust virtual try-on. The
main challenge lies in retaining the intricate textures of the garment while
maintaining the flexibility of pre-trained Stable Diffusion. Our solution
involves the development of a garment encoder, a trainable copy of the
denoising UNet equipped with additive self-attention (ASA) layers. These ASA
layers are specifically devised to transfer detailed garment textures, also
facilitating the integration of stylized base models for the creation of
stylized images. Furthermore, the incorporation of a dedicated try-on
ControlNet enables StableGarment to execute virtual try-on tasks with
precision. We also build a novel data engine that produces high-quality
synthesized data to preserve the model's ability to follow prompts. Extensive
experiments demonstrate that our approach delivers state-of-the-art (SOTA)
results among existing virtual try-on methods and exhibits high flexibility
with broad potential applications in various garment-centric image generation.