Can Biases in ImageNet Models Explain Generalization?
Paul Gavrikov Janis Keuper
The robust generalization of models to rare, in-distribution (ID) samples
drawn from the long tail of the training distribution and to
out-of-training-distribution (OOD) samples is one of the major challenges of
current deep learning methods. For image classification, this manifests in the
existence of adversarial attacks, the performance drops on distorted images,
and a lack of generalization to concepts such as sketches. The current
understanding of generalization in neural networks is very limited, but some
biases that differentiate models from human vision have been identified and
might be causing these limitations. Consequently, several attempts with varying
success have been made to reduce these biases during training to improve
generalization. We take a step back and sanity-check these attempts. Fixing the
architecture to the well-established ResNet-50, we perform a large-scale study
on 48 ImageNet models obtained via different training methods to understand how
and if these biases - including shape bias, spectral biases, and critical bands
- interact with generalization. Our extensive study results reveal that
contrary to previous findings, these biases are insufficient to accurately
predict the generalization of a model holistically. We provide access to all
checkpoints and evaluation code at
https://github.com/paulgavrikov/biases_vs_generalization