You Only Condense Once: Two Rules for Pruning Condensed Datasets
Yang He Lingao Xiao Joey Tianyi Zhou
Dataset condensation is a crucial tool for enhancing training efficiency by
reducing the size of the training dataset, particularly in on-device scenarios.
However, these scenarios have two significant challenges: 1) the varying
computational resources available on the devices require a dataset size
different from the pre-defined condensed dataset, and 2) the limited
computational resources often preclude the possibility of conducting additional
condensation processes. We introduce You Only Condense Once (YOCO) to overcome
these limitations. On top of one condensed dataset, YOCO produces smaller
condensed datasets with two embarrassingly simple dataset pruning rules: Low
LBPE Score and Balanced Construction. YOCO offers two key advantages: 1) it can
flexibly resize the dataset to fit varying computational constraints, and 2) it
eliminates the need for extra condensation processes, which can be
computationally prohibitive. Experiments validate our findings on networks
including ConvNet, ResNet and DenseNet, and datasets including CIFAR-10,
CIFAR-100 and ImageNet. For example, our YOCO surpassed various dataset
condensation and dataset pruning methods on CIFAR-10 with ten Images Per Class
(IPC), achieving 6.98-8.89% and 6.31-23.92% accuracy gains, respectively. The
code is available at: https://github.com/he-y/you-only-condense-once.