Decoupled Contrastive Learning for Long-Tailed Recognition
Shiyu Xuan Shiliang Zhang
Supervised Contrastive Loss (SCL) is popular in visual representation
learning. Given an anchor image, SCL pulls two types of positive samples, i.e.,
its augmentation and other images from the same class together, while pushes
negative images apart to optimize the learned embedding. In the scenario of
long-tailed recognition, where the number of samples in each class is
imbalanced, treating two types of positive samples equally leads to the biased
optimization for intra-category distance. In addition, similarity relationship
among negative samples, that are ignored by SCL, also presents meaningful
semantic cues. To improve the performance on long-tailed recognition, this
paper addresses those two issues of SCL by decoupling the training objective.
Specifically, it decouples two types of positives in SCL and optimizes their
relations toward different objectives to alleviate the influence of the
imbalanced dataset. We further propose a patch-based self distillation to
transfer knowledge from head to tail classes to relieve the
under-representation of tail classes. It uses patch-based features to mine
shared visual patterns among different instances and leverages a self
distillation procedure to transfer such knowledge. Experiments on different
long-tailed classification benchmarks demonstrate the superiority of our
method. For instance, it achieves the 57.7% top-1 accuracy on the ImageNet-LT
dataset. Combined with the ensemble-based method, the performance can be
further boosted to 59.7%, which substantially outperforms many recent works.
The code is available at https://github.com/SY-Xuan/DSCL.