Robust Long-Tailed Learning via Label-Aware Bounded CVaR
Hong Zhu Runpeng Yu Xing Tang Yifei Wang Yuan Fang Yisen Wang
Data in the real-world classification problems are always imbalanced or
long-tailed, wherein the majority classes have the most of the samples that
dominate the model training. In such setting, the naive model tends to have
poor performance on the minority classes. Previously, a variety of loss
modifications have been proposed to address the long-tailed leaning problem,
while these methods either treat the samples in the same class
indiscriminatingly or lack a theoretical guarantee. In this paper, we propose
two novel approaches based on CVaR (Conditional Value at Risk) to improve the
performance of long-tailed learning with a solid theoretical ground.
Specifically, we firstly introduce a Label-Aware Bounded CVaR (LAB-CVaR) loss
to overcome the pessimistic result of the original CVaR, and further design the
optimal weight bounds for LAB-CVaR theoretically. Based on LAB-CVaR, we
additionally propose a LAB-CVaR with logit adjustment (LAB-CVaR-logit) loss to
stabilize the optimization process, where we also offer the theoretical
support. Extensive experiments on real-world datasets with long-tailed label
distributions verify the superiority of our proposed methods.