Semantic Generative Augmentations for Few-Shot Counting
Perla Doubinsky Nicolas Audebert Michel Crucianu Hervé Le Borgne
With the availability of powerful text-to-image diffusion models, recent
works have explored the use of synthetic data to improve image classification
performances. These works show that it can effectively augment or even replace
real data. In this work, we investigate how synthetic data can benefit few-shot
class-agnostic counting. This requires to generate images that correspond to a
given input number of objects. However, text-to-image models struggle to grasp
the notion of count. We propose to rely on a double conditioning of Stable
Diffusion with both a prompt and a density map in order to augment a training
dataset for few-shot counting. Due to the small dataset size, the fine-tuned
model tends to generate images close to the training images. We propose to
enhance the diversity of synthesized images by exchanging captions between
images thus creating unseen configurations of object types and spatial layout.
Our experiments show that our diversified generation strategy significantly
improves the counting accuracy of two recent and performing few-shot counting
models on FSC147 and CARPK.