Combating Representation Learning Disparity with Geometric Harmonization
Zhihan Zhou Jiangchao Yao Feng Hong Ya Zhang Bo Han Yanfeng Wang
Self-supervised learning (SSL) as an effective paradigm of representation
learning has achieved tremendous success on various curated datasets in diverse
scenarios. Nevertheless, when facing the long-tailed distribution in real-world
applications, it is still hard for existing methods to capture transferable and
robust representation. Conventional SSL methods, pursuing sample-level
uniformity, easily leads to representation learning disparity where head
classes dominate the feature regime but tail classes passively collapse. To
address this problem, we propose a novel Geometric Harmonization (GH) method to
encourage category-level uniformity in representation learning, which is more
benign to the minority and almost does not hurt the majority under long-tailed
distribution. Specially, GH measures the population statistics of the embedding
space on top of self-supervised learning, and then infer an fine-grained
instance-wise calibration to constrain the space expansion of head classes and
avoid the passive collapse of tail classes. Our proposal does not alter the
setting of SSL and can be easily integrated into existing methods in a low-cost
manner. Extensive results on a range of benchmark datasets show the
effectiveness of GH with high tolerance to the distribution skewness. Our code
is available at https://github.com/MediaBrain-SJTU/Geometric-Harmonization.