OMG: Occlusion-friendly Personalized Multi-concept Generation in
  Diffusion Models
Zhe Kong Yong Zhang Tianyu Yang Tao Wang Kaihao Zhang Bizhu Wu Guanying Chen Wei Liu Wenhan Luo
Personalization is an important topic in text-to-image generation, especially
the challenging multi-concept personalization. Current multi-concept methods
are struggling with identity preservation, occlusion, and the harmony between
foreground and background. In this work, we propose OMG, an occlusion-friendly
personalized generation framework designed to seamlessly integrate multiple
concepts within a single image. We propose a novel two-stage sampling solution.
The first stage takes charge of layout generation and visual comprehension
information collection for handling occlusions. The second one utilizes the
acquired visual comprehension information and the designed noise blending to
integrate multiple concepts while considering occlusions. We also observe that
the initiation denoising timestep for noise blending is the key to identity
preservation and layout. Moreover, our method can be combined with various
single-concept models, such as LoRA and InstantID without additional tuning.
Especially, LoRA models on civitai.com can be exploited directly. Extensive
experiments demonstrate that OMG exhibits superior performance in multi-concept
personalization.