Asymmetric Patch Sampling for Contrastive Learning
Chengchao Shen Jianzhong Chen Shu Wang Hulin Kuang Jin Liu Jianxin Wang
Asymmetric appearance between positive pair effectively reduces the risk of
representation degradation in contrastive learning. However, there are still a
mass of appearance similarities between positive pair constructed by the
existing methods, which inhibits the further representation improvement. In
this paper, we propose a novel asymmetric patch sampling strategy for
contrastive learning, to further boost the appearance asymmetry for better
representations. Specifically, dual patch sampling strategies are applied to
the given image, to obtain asymmetric positive pairs. First, sparse patch
sampling is conducted to obtain the first view, which reduces spatial
redundancy of image and allows a more asymmetric view. Second, a selective
patch sampling is proposed to construct another view with large appearance
discrepancy relative to the first one. Due to the inappreciable appearance
similarity between positive pair, the trained model is encouraged to capture
the similarity on semantics, instead of low-level ones. Experimental results
demonstrate that our proposed method significantly outperforms the existing
self-supervised methods on both ImageNet-1K and CIFAR dataset, e.g., 2.5%
finetune accuracy improvement on CIFAR100. Furthermore, our method achieves
state-of-the-art performance on downstream tasks, object detection and instance
segmentation on COCO.Additionally, compared to other self-supervised methods,
our method is more efficient on both memory and computation during training.
The source code is available at https://github.com/visresearch/aps.