Portrait Diffusion: Training-free Face Stylization with
  Chain-of-Painting
Jin Liu Huaibo Huang Chao Jin Ran He
Face stylization refers to the transformation of a face into a specific
portrait style. However, current methods require the use of example-based
adaptation approaches to fine-tune pre-trained generative models so that they
demand lots of time and storage space and fail to achieve detailed style
transformation. This paper proposes a training-free face stylization framework,
named Portrait Diffusion. This framework leverages off-the-shelf text-to-image
diffusion models, eliminating the need for fine-tuning specific examples.
Specifically, the content and style images are first inverted into latent
codes. Then, during image reconstruction using the corresponding latent code,
the content and style features in the attention space are delicately blended
through a modified self-attention operation called Style Attention Control.
Additionally, a Chain-of-Painting method is proposed for the gradual redrawing
of unsatisfactory areas from rough adjustments to fine-tuning. Extensive
experiments validate the effectiveness of our Portrait Diffusion method and
demonstrate the superiority of Chain-of-Painting in achieving precise face
stylization. Code will be released at
\url{https://github.com/liujin112/PortraitDiffusion}.