Coordinated Sparse Recovery of Label Noise
Yukun Yang Naihao Wang Haixin Yang Ruirui Li
Label noise is a common issue in real-world datasets that inevitably impacts
the generalization of models. This study focuses on robust classification tasks
where the label noise is instance-dependent. Estimating the transition matrix
accurately in this task is challenging, and methods based on sample selection
often exhibit confirmation bias to varying degrees. Sparse over-parameterized
training (SOP) has been theoretically effective in estimating and recovering
label noise, offering a novel solution for noise-label learning. However, this
study empirically observes and verifies a technical flaw of SOP: the lack of
coordination between model predictions and noise recovery leads to increased
generalization error. To address this, we propose a method called Coordinated
Sparse Recovery (CSR). CSR introduces a collaboration matrix and confidence
weights to coordinate model predictions and noise recovery, reducing error
leakage. Based on CSR, this study designs a joint sample selection strategy and
constructs a comprehensive and powerful learning framework called CSR+. CSR+
significantly reduces confirmation bias, especially for datasets with more
classes and a high proportion of instance-specific noise. Experimental results
on simulated and real-world noisy datasets demonstrate that both CSR and CSR+
achieve outstanding performance compared to methods at the same level.