Learning Disentangled Prompts for Compositional Image Synthesis
Kihyuk Sohn Albert Shaw Yuan Hao Han Zhang Luisa Polania Huiwen Chang Lu Jiang Irfan Essa
We study domain-adaptive image synthesis, the problem of teaching pretrained
image generative models a new style or concept from as few as one image to
synthesize novel images, to better understand the compositional image
synthesis. We present a framework that leverages a pretrained class-conditional
generation model and visual prompt tuning. Specifically, we propose a novel
source class distilled visual prompt that learns disentangled prompts of
semantic (e.g., class) and domain (e.g., style) from a few images. Learned
domain prompt is then used to synthesize images of any classes in the style of
target domain. We conduct studies on various target domains with the number of
images ranging from one to a few to many, and show qualitative results which
show the compositional generalization of our method. Moreover, we show that our
method can help improve zero-shot domain adaptation classification accuracy.