Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models
Gihyun Kwon Simon Jenni Dingzeyu Li Joon-Young Lee Jong Chul Ye Fabian Caba Heilbron
While there has been significant progress in customizing text-to-image
generation models, generating images that combine multiple personalized
concepts remains challenging. In this work, we introduce Concept Weaver, a
method for composing customized text-to-image diffusion models at inference
time. Specifically, the method breaks the process into two steps: creating a
template image aligned with the semantics of input prompts, and then
personalizing the template using a concept fusion strategy. The fusion strategy
incorporates the appearance of the target concepts into the template image
while retaining its structural details. The results indicate that our method
can generate multiple custom concepts with higher identity fidelity compared to
alternative approaches. Furthermore, the method is shown to seamlessly handle
more than two concepts and closely follow the semantic meaning of the input
prompt without blending appearances across different subjects.