GenView: Enhancing View Quality with Pretrained Generative Model for
  Self-Supervised Learning
Xiaojie Li Yibo Yang Xiangtai Li Jianlong Wu Yue Yu Bernard Ghanem Min Zhang
Self-supervised learning has achieved remarkable success in acquiring
high-quality representations from unlabeled data. The widely adopted
contrastive learning framework aims to learn invariant representations by
minimizing the distance between positive views originating from the same image.
However, existing techniques to construct positive views highly rely on manual
transformations, resulting in limited diversity and potentially false positive
pairs. To tackle these challenges, we present GenView, a controllable framework
that augments the diversity of positive views leveraging the power of
pretrained generative models while preserving semantics. We develop an adaptive
view generation method that dynamically adjusts the noise level in sampling to
ensure the preservation of essential semantic meaning while introducing
variability. Additionally, we introduce a quality-driven contrastive loss,
which assesses the quality of positive pairs by considering both foreground
similarity and background diversity. This loss prioritizes the high-quality
positive pairs we construct while reducing the influence of low-quality pairs,
thereby mitigating potential semantic inconsistencies introduced by generative
models and aggressive data augmentation. Thanks to the improved positive view
quality and the quality-driven contrastive loss, GenView significantly improves
self-supervised learning across various tasks. For instance, GenView improves
MoCov2 performance by 2.5%/2.2% on ImageNet linear/semi-supervised
classification. Moreover, GenView even performs much better than naively
augmenting the ImageNet dataset with Laion400M or ImageNet21K. Code is
available at https://github.com/xiaojieli0903/genview.