DreamInpainter: Text-Guided Subject-Driven Image Inpainting with
  Diffusion Models
Shaoan Xie Yang Zhao Zhisheng Xiao Kelvin C. K. Chan Yandong Li Yanwu Xu Kun Zhang Tingbo Hou
This study introduces Text-Guided Subject-Driven Image Inpainting, a novel
task that combines text and exemplar images for image inpainting. While both
text and exemplar images have been used independently in previous efforts,
their combined utilization remains unexplored. Simultaneously accommodating
both conditions poses a significant challenge due to the inherent balance
required between editability and subject fidelity. To tackle this challenge, we
propose a two-step approach DreamInpainter. First, we compute dense subject
features to ensure accurate subject replication. Then, we employ a
discriminative token selection module to eliminate redundant subject details,
preserving the subject's identity while allowing changes according to other
conditions such as mask shape and text prompts. Additionally, we introduce a
decoupling regularization technique to enhance text control in the presence of
exemplar images. Our extensive experiments demonstrate the superior performance
of our method in terms of visual quality, identity preservation, and text
control, showcasing its effectiveness in the context of text-guided
subject-driven image inpainting.