BATINet: Background-Aware Text to Image Synthesis and Manipulation
  Network
Ryugo Morita Zhiqiang Zhang Jinjia Zhou
Background-Induced Text2Image (BIT2I) aims to generate foreground content
according to the text on the given background image. Most studies focus on
generating high-quality foreground content, although they ignore the
relationship between the two contents. In this study, we analyzed a novel
Background-Aware Text2Image (BAT2I) task in which the generated content matches
the input background. We proposed a Background-Aware Text to Image synthesis
and manipulation Network (BATINet), which contains two key components: Position
Detect Network (PDN) and Harmonize Network (HN). The PDN detects the most
plausible position of the text-relevant object in the background image. The HN
harmonizes the generated content referring to background style information.
Finally, we reconstructed the generation network, which consists of the
multi-GAN and attention module to match more user preferences. Moreover, we can
apply BATINet to text-guided image manipulation. It solves the most challenging
task of manipulating the shape of an object. We demonstrated through
qualitative and quantitative evaluations on the CUB dataset that the proposed
model outperforms other state-of-the-art methods.