Feature Noise Boosts DNN Generalization under Label Noise
Lu Zeng Xuan Chen Xiaoshuang Shi Heng Tao Shen
The presence of label noise in the training data has a profound impact on the
generalization of deep neural networks (DNNs). In this study, we introduce and
theoretically demonstrate a simple feature noise method, which directly adds
noise to the features of training data, can enhance the generalization of DNNs
under label noise. Specifically, we conduct theoretical analyses to reveal that
label noise leads to weakened DNN generalization by loosening the PAC-Bayes
generalization bound, and feature noise results in better DNN generalization by
imposing an upper bound on the mutual information between the model weights and
the features, which constrains the PAC-Bayes generalization bound. Furthermore,
to ensure effective generalization of DNNs in the presence of label noise, we
conduct application analyses to identify the optimal types and levels of
feature noise to add for obtaining desirable label noise generalization.
Finally, extensive experimental results on several popular datasets demonstrate
the feature noise method can significantly enhance the label noise
generalization of the state-of-the-art label noise method.