Measuring Diversity in Co-creative Image Generation
Francisco Ibarrola Kazjon Grace
Quality and diversity have been proposed as reasonable heuristics for
assessing content generated by co-creative systems, but to date there has been
little agreement around what constitutes the latter or how to measure it.
Proposed approaches for assessing generative models in terms of diversity have
limitations in that they compare the model's outputs to a ground truth that in
the era of large pre-trained generative models might not be available, or
entail an impractical number of computations. We propose an alternative based
on entropy of neural network encodings for comparing diversity between sets of
images that does not require ground-truth knowledge and is easy to compute. We
also compare two pre-trained networks and show how the choice relates to the
notion of diversity that we want to evaluate. We conclude with a discussion of
the potential applications of these measures for ideation in interactive
systems, model evaluation, and more broadly within computational creativity.