PLReMix: Combating Noisy Labels with Pseudo-Label Relaxed Contrastive
  Representation Learning
Xiaoyu Liu Beitong Zhou Cheng Cheng
Recently, the application of Contrastive Representation Learning (CRL) in
learning with noisy labels (LNL) has shown promising advancements due to its
remarkable ability to learn well-distributed representations for better
distinguishing noisy labels. However, CRL is mainly used as a pre-training
technique, leading to a complicated multi-stage training pipeline. We also
observed that trivially combining CRL with supervised LNL methods decreases
performance. Using different images from the same class as negative pairs in
CRL creates optimization conflicts between CRL and the supervised loss. To
address these two issues, we propose an end-to-end PLReMix framework that
avoids the complicated pipeline by introducing a Pseudo-Label Relaxed (PLR)
contrastive loss to alleviate the conflicts between losses. This PLR loss
constructs a reliable negative set of each sample by filtering out its
inappropriate negative pairs that overlap at the top k indices of prediction
probabilities, leading to more compact semantic clusters than vanilla CRL.
Furthermore, a two-dimensional Gaussian Mixture Model (GMM) is adopted to
distinguish clean and noisy samples by leveraging semantic information and
model outputs simultaneously, which is expanded on the previously widely used
one-dimensional form. The PLR loss and a semi-supervised loss are
simultaneously applied to train on the GMM divided clean and noisy samples.
Experiments on multiple benchmark datasets demonstrate the effectiveness of the
proposed method. Our proposed PLR loss is scalable, which can be easily
integrated into other LNL methods and boost their performance. Codes will be
available.