Debiasing, calibrating, and improving Semi-supervised Learning
  performance via simple Ensemble Projector
Khanh-Binh Nguyen
Recent studies on semi-supervised learning (SSL) have achieved great success.
Despite their promising performance, current state-of-the-art methods tend
toward increasingly complex designs at the cost of introducing more network
components and additional training procedures. In this paper, we propose a
simple method named Ensemble Projectors Aided for Semi-supervised Learning
(EPASS), which focuses mainly on improving the learned embeddings to boost the
performance of the existing contrastive joint-training semi-supervised learning
frameworks. Unlike standard methods, where the learned embeddings from one
projector are stored in memory banks to be used with contrastive learning,
EPASS stores the ensemble embeddings from multiple projectors in memory banks.
As a result, EPASS improves generalization, strengthens feature representation,
and boosts performance. For instance, EPASS improves strong baselines for
semi-supervised learning by 39.47\%/31.39\%/24.70\% top-1 error rate, while
using only 100k/1\%/10\% of labeled data for SimMatch, and achieves
40.24\%/32.64\%/25.90\% top-1 error rate for CoMatch on the ImageNet dataset.
These improvements are consistent across methods, network architectures, and
datasets, proving the general effectiveness of the proposed methods. Code is
available at https://github.com/beandkay/EPASS.