Desigen: A Pipeline for Controllable Design Template Generation
Haohan Weng Danqing Huang Yu Qiao Zheng Hu Chin-Yew Lin Tong Zhang C. L. Philip Chen
Templates serve as a good starting point to implement a design (e.g., banner,
slide) but it takes great effort from designers to manually create. In this
paper, we present Desigen, an automatic template creation pipeline which
generates background images as well as harmonious layout elements over the
background. Different from natural images, a background image should preserve
enough non-salient space for the overlaying layout elements. To equip existing
advanced diffusion-based models with stronger spatial control, we propose two
simple but effective techniques to constrain the saliency distribution and
reduce the attention weight in desired regions during the background generation
process. Then conditioned on the background, we synthesize the layout with a
Transformer-based autoregressive generator. To achieve a more harmonious
composition, we propose an iterative inference strategy to adjust the
synthesized background and layout in multiple rounds. We constructed a design
dataset with more than 40k advertisement banners to verify our approach.
Extensive experiments demonstrate that the proposed pipeline generates
high-quality templates comparable to human designers. More than a single-page
design, we further show an application of presentation generation that outputs
a set of theme-consistent slides. The data and code are available at
https://whaohan.github.io/desigen.