Learning Label Hierarchy with Supervised Contrastive Learning
Ruixue Lian William A. Sethares Junjie Hu
Supervised contrastive learning (SCL) frameworks treat each class as
independent and thus consider all classes to be equally important. This
neglects the common scenario in which label hierarchy exists, where
fine-grained classes under the same category show more similarity than very
different ones. This paper introduces a family of Label-Aware SCL methods
(LASCL) that incorporates hierarchical information to SCL by leveraging
similarities between classes, resulting in creating a more well-structured and
discriminative feature space. This is achieved by first adjusting the distance
between instances based on measures of the proximity of their classes with the
scaled instance-instance-wise contrastive. An additional instance-center-wise
contrastive is introduced to move within-class examples closer to their
centers, which are represented by a set of learnable label parameters. The
learned label parameters can be directly used as a nearest neighbor classifier
without further finetuning. In this way, a better feature representation is
generated with improvements of intra-cluster compactness and inter-cluster
separation. Experiments on three datasets show that the proposed LASCL works
well on text classification of distinguishing a single label among
multi-labels, outperforming the baseline supervised approaches. Our code is
publicly available.