Pseudo-label Correction for Instance-dependent Noise Using
  Teacher-student Framework
Eugene Kim
The high capacity of deep learning models to learn complex patterns poses a
significant challenge when confronted with label noise. The inability to
differentiate clean and noisy labels ultimately results in poor generalization.
We approach this problem by reassigning the label for each image using a new
teacher-student based framework termed P-LC (pseudo-label correction).
Traditional teacher-student networks are composed of teacher and student
classifiers for knowledge distillation. In our novel approach, we reconfigure
the teacher network into a triple encoder, leveraging the triplet loss to
establish a pseudo-label correction system. As the student generates pseudo
labels for a set of given images, the teacher learns to choose between the
initially assigned labels and the pseudo labels. Experiments on MNIST,
Fashion-MNIST, and SVHN demonstrate P-LC's superior performance over existing
state-of-the-art methods across all noise levels, most notably in high noise.
In addition, we introduce a noise level estimation to help assess model
performance and inform the need for additional data cleaning procedures.