The Challenges of Image Generation Models in Generating Multi-Component
  Images
Tham Yik Foong Shashank Kotyan Po Yuan Mao Danilo Vasconcellos Vargas
Recent advances in text-to-image generators have led to substantial
capabilities in image generation. However, the complexity of prompts acts as a
bottleneck in the quality of images generated. A particular under-explored
facet is the ability of generative models to create high-quality images
comprising multiple components given as a prior. In this paper, we propose and
validate a metric called Components Inclusion Score (CIS) to evaluate the
extent to which a model can correctly generate multiple components. Our results
reveal that the evaluated models struggle to incorporate all the visual
elements from prompts with multiple components (8.53% drop in CIS per component
for all evaluated models). We also identify a significant decline in the
quality of the images and context awareness within an image as the number of
components increased (15.91% decrease in inception Score and 9.62% increase in
Frechet Inception Distance). To remedy this issue, we fine-tuned Stable
Diffusion V2 on a custom-created test dataset with multiple components,
outperforming its vanilla counterpart. To conclude, these findings reveal a
critical limitation in existing text-to-image generators, shedding light on the
challenge of generating multiple components within a single image using a
complex prompt.