Inferring and Leveraging Parts from Object Shape for Improving Semantic
  Image Synthesis
Yuxiang Wei Zhilong Ji Xiaohe Wu Jinfeng Bai Lei Zhang Wangmeng Zuo
Despite the progress in semantic image synthesis, it remains a challenging
problem to generate photo-realistic parts from input semantic map. Integrating
part segmentation map can undoubtedly benefit image synthesis, but is
bothersome and inconvenient to be provided by users. To improve part synthesis,
this paper presents to infer Parts from Object ShapE (iPOSE) and leverage it
for improving semantic image synthesis. However, albeit several part
segmentation datasets are available, part annotations are still not provided
for many object categories in semantic image synthesis. To circumvent it, we
resort to few-shot regime to learn a PartNet for predicting the object part map
with the guidance of pre-defined support part maps. PartNet can be readily
generalized to handle a new object category when a small number (e.g., 3) of
support part maps for this category are provided. Furthermore, part semantic
modulation is presented to incorporate both inferred part map and semantic map
for image synthesis. Experiments show that our iPOSE not only generates objects
with rich part details, but also enables to control the image synthesis
flexibly. And our iPOSE performs favorably against the state-of-the-art methods
in terms of quantitative and qualitative evaluation. Our code will be publicly
available at https://github.com/csyxwei/iPOSE.