Large-scale Dataset Pruning with Dynamic Uncertainty
Muyang He Shuo Yang Tiejun Huang Bo Zhao
The state of the art of many learning tasks, e.g., image classification, is
advanced by collecting larger datasets and then training larger models on them.
As the outcome, the increasing computational cost is becoming unaffordable. In
this paper, we investigate how to prune the large-scale datasets, and thus
produce an informative subset for training sophisticated deep models with
negligible performance drop. We propose a simple yet effective dataset pruning
method by exploring both the prediction uncertainty and training dynamics. To
our knowledge, this is the first work to study dataset pruning on large-scale
datasets, i.e., ImageNet-1K and ImageNet-21K, and advanced models, i.e., Swin
Transformer and ConvNeXt. Extensive experimental results indicate that our
method outperforms the state of the art and achieves 75% lossless compression
ratio on both ImageNet-1K and ImageNet-21K. The code and pruned datasets are
available at https://github.com/BAAI-DCAI/Dataset-Pruning.