Feature Space Renormalization for Semi-supervised Learning
Jun Sun Zhongjie Mao Chao Li Chao Zhou Xiao-Jun Wu
Semi-supervised learning (SSL) has been proven to be a powerful method for
leveraging unlabelled data to alleviate models' dependence on large labelled
datasets. The common framework among recent approaches is to train the model on
a large amount of unlabelled data with consistency regularization to constrain
the model predictions to be invariant to input perturbation. However, the
existing SSL frameworks still have room for improvement in the consistency
regularization method. Instead of regularizing category predictions in the
label space as in existing frameworks, this paper proposes a feature space
renormalization (FSR) mechanism for SSL. First, we propose a feature space
renormalization mechanism to substitute for the commonly used consistency
regularization mechanism to learn better discriminative features. To apply this
mechanism, we start by building a basic model and an empirical model and then
introduce our mechanism to renormalize the feature learning of the basic model
with the guidance of the empirical model. Second, we combine the proposed
mechanism with pseudo-labelling to obtain a novel effective SSL model named
FreMatch. The experimental results show that our method can achieve better
performance on a variety of standard SSL benchmark datasets, and the proposed
feature space renormalization mechanism can also enhance the performance of
other SSL approaches.