One Category One Prompt: Dataset Distillation using Diffusion Models
Ali Abbasi Ashkan Shahbazi Hamed Pirsiavash Soheil Kolouri
The extensive amounts of data required for training deep neural networks pose
significant challenges on storage and transmission fronts. Dataset distillation
has emerged as a promising technique to condense the information of massive
datasets into a much smaller yet representative set of synthetic samples.
However, traditional dataset distillation approaches often struggle to scale
effectively with high-resolution images and more complex architectures due to
the limitations in bi-level optimization. Recently, several works have proposed
exploiting knowledge distillation with decoupled optimization schemes to scale
up dataset distillation. Although these methods effectively address the
scalability issue, they rely on extensive image augmentations requiring the
storage of soft labels for augmented images. In this paper, we introduce
Dataset Distillation using Diffusion Models (D3M) as a novel paradigm for
dataset distillation, leveraging recent advancements in generative
text-to-image foundation models. Our approach utilizes textual inversion, a
technique for fine-tuning text-to-image generative models, to create concise
and informative representations for large datasets. By employing these learned
text prompts, we can efficiently store and infer new samples for introducing
data variability within a fixed memory budget. We show the effectiveness of our
method through extensive experiments across various computer vision benchmark
datasets with different memory budgets.