Tied-Augment: Controlling Representation Similarity Improves Data
  Augmentation
Emirhan Kurtulus Zichao Li Yann Dauphin Ekin Dogus Cubuk
Data augmentation methods have played an important role in the recent advance
of deep learning models, and have become an indispensable component of
state-of-the-art models in semi-supervised, self-supervised, and supervised
training for vision. Despite incurring no additional latency at test time, data
augmentation often requires more epochs of training to be effective. For
example, even the simple flips-and-crops augmentation requires training for
more than 5 epochs to improve performance, whereas RandAugment requires more
than 90 epochs. We propose a general framework called Tied-Augment, which
improves the efficacy of data augmentation in a wide range of applications by
adding a simple term to the loss that can control the similarity of
representations under distortions. Tied-Augment can improve state-of-the-art
methods from data augmentation (e.g. RandAugment, mixup), optimization (e.g.
SAM), and semi-supervised learning (e.g. FixMatch). For example,
Tied-RandAugment can outperform RandAugment by 2.0% on ImageNet. Notably, using
Tied-Augment, data augmentation can be made to improve generalization even when
training for a few epochs and when fine-tuning. We open source our code at
https://github.com/ekurtulus/tied-augment/tree/main.