Towards Controllable Face Generation with Semantic Latent Diffusion
  Models
Alex Ergasti Claudio Ferrari Tomaso Fontanini Massimo Bertozzi Andrea Prati
Semantic Image Synthesis (SIS) is among the most popular and effective
techniques in the field of face generation and editing, thanks to its good
generation quality and the versatility is brings along. Recent works attempted
to go beyond the standard GAN-based framework, and started to explore Diffusion
Models (DMs) for this task as these stand out with respect to GANs in terms of
both quality and diversity. On the other hand, DMs lack in fine-grained
controllability and reproducibility. To address that, in this paper we propose
a SIS framework based on a novel Latent Diffusion Model architecture for human
face generation and editing that is both able to reproduce and manipulate a
real reference image and generate diversity-driven results. The proposed system
utilizes both SPADE normalization and cross-attention layers to merge shape and
style information and, by doing so, allows for a precise control over each of
the semantic parts of the human face. This was not possible with previous
methods in the state of the art. Finally, we performed an extensive set of
experiments to prove that our model surpasses current state of the art, both
qualitatively and quantitatively.