Understanding the Benefits of Image Augmentations
Matthew Iceland Christopher Kanan
Image Augmentations are widely used to reduce overfitting in neural networks.
However, the explainability of their benefits largely remains a mystery. We
study which layers of residual neural networks (ResNets) are most affected by
augmentations using Centered Kernel Alignment (CKA). We do so by analyzing
models of varying widths and depths, as well as whether their weights are
initialized randomly or through transfer learning. We find that the pattern of
how the layers are affected depends on the model's depth, and that networks
trained with augmentation that use information from two images affect the
learned weights significantly more than augmentations that operate on a single
image. Deeper layers of ResNets initialized with ImageNet-1K weights and
fine-tuned receive more impact from the augmentations than early layers.
Understanding the effects of image augmentations on CNNs will have a variety of
applications, such as determining how far back one needs to fine-tune a network
and which layers should be frozen when implementing layer freezing algorithms.