Poly-View Contrastive Learning
Amitis Shidani Devon Hjelm Jason Ramapuram Russ Webb Eeshan Gunesh Dhekane Dan Busbridge
Contrastive learning typically matches pairs of related views among a number
of unrelated negative views. Views can be generated (e.g. by augmentations) or
be observed. We investigate matching when there are more than two related views
which we call poly-view tasks, and derive new representation learning
objectives using information maximization and sufficient statistics. We show
that with unlimited computation, one should maximize the number of related
views, and with a fixed compute budget, it is beneficial to decrease the number
of unique samples whilst increasing the number of views of those samples. In
particular, poly-view contrastive models trained for 128 epochs with batch size
256 outperform SimCLR trained for 1024 epochs at batch size 4096 on ImageNet1k,
challenging the belief that contrastive models require large batch sizes and
many training epochs.