Pi-DUAL: Using Privileged Information to Distinguish Clean from Noisy
  Labels
Ke Wang Guillermo Ortiz-Jimenez Rodolphe Jenatton Mark Collier Efi Kokiopoulou Pascal Frossard
Label noise is a pervasive problem in deep learning that often compromises
the generalization performance of trained models. Recently, leveraging
privileged information (PI) -- information available only during training but
not at test time -- has emerged as an effective approach to mitigate this
issue. Yet, existing PI-based methods have failed to consistently outperform
their no-PI counterparts in terms of preventing overfitting to label noise. To
address this deficiency, we introduce Pi-DUAL, an architecture designed to
harness PI to distinguish clean from wrong labels. Pi-DUAL decomposes the
output logits into a prediction term, based on conventional input features, and
a noise-fitting term influenced solely by PI. A gating mechanism steered by PI
adaptively shifts focus between these terms, allowing the model to implicitly
separate the learning paths of clean and wrong labels. Empirically, Pi-DUAL
achieves significant performance improvements on key PI benchmarks (e.g., +6.8%
on ImageNet-PI), establishing a new state-of-the-art test set accuracy.
Additionally, Pi-DUAL is a potent method for identifying noisy samples
post-training, outperforming other strong methods at this task. Overall,
Pi-DUAL is a simple, scalable and practical approach for mitigating the effects
of label noise in a variety of real-world scenarios with PI.