Generate E-commerce Product Background by Integrating Category
  Commonality and Personalized Style
Haohan Wang Wei Feng Yang Lu Yaoyu Li Zheng Zhang Jingjing Lv Xin Zhu Junjie Shen Zhangang Lin Lixing Bo Jingping Shao
The state-of-the-art methods for e-commerce product background generation
suffer from the inefficiency of designing product-wise prompts when scaling up
the production, as well as the ineffectiveness of describing fine-grained
styles when customizing personalized backgrounds for some specific brands. To
address these obstacles, we integrate the category commonality and personalized
style into diffusion models. Concretely, we propose a Category-Wise Generator
to enable large-scale background generation for the first time. A unique
identifier in the prompt is assigned to each category, whose attention is
located on the background by a mask-guided cross attention layer to learn the
category-wise style. Furthermore, for products with specific and fine-grained
requirements in layout, elements, etc, a Personality-Wise Generator is devised
to learn such personalized style directly from a reference image to resolve
textual ambiguities, and is trained in a self-supervised manner for more
efficient training data usage. To advance research in this field, the first
large-scale e-commerce product background generation dataset BG60k is
constructed, which covers more than 60k product images from over 2k categories.
Experiments demonstrate that our method could generate high-quality backgrounds
for different categories, and maintain the personalized background style of
reference images. The link to BG60k and codes will be available soon.