DiffuLT: How to Make Diffusion Model Useful for Long-tail Recognition
Jie Shao Ke Zhu Hanxiao Zhang Jianxin Wu
This paper proposes a new pipeline for long-tail (LT) recognition. Instead of
re-weighting or re-sampling, we utilize the long-tailed dataset itself to
generate a balanced proxy that can be optimized through cross-entropy (CE).
Specifically, a randomly initialized diffusion model, trained exclusively on
the long-tailed dataset, is employed to synthesize new samples for
underrepresented classes. Then, we utilize the inherent information in the
original dataset to filter out harmful samples and keep the useful ones. Our
strategy, Diffusion model for Long-Tail recognition (DiffuLT), represents a
pioneering utilization of generative models in long-tail recognition. DiffuLT
achieves state-of-the-art results on CIFAR10-LT, CIFAR100-LT, and ImageNet-LT,
surpassing the best competitors with non-trivial margins. Abundant ablations
make our pipeline interpretable, too. The whole generation pipeline is done
without any external data or pre-trained model weights, making it highly
generalizable to real-world long-tailed settings.