On the Foundations of Shortcut Learning
Katherine L. Hermann Hossein Mobahi Thomas Fel Michael C. Mozer
Deep-learning models can extract a rich assortment of features from data.
Which features a model uses depends not only on predictivity-how reliably a
feature indicates train-set labels-but also on availability-how easily the
feature can be extracted, or leveraged, from inputs. The literature on shortcut
learning has noted examples in which models privilege one feature over another,
for example texture over shape and image backgrounds over foreground objects.
Here, we test hypotheses about which input properties are more available to a
model, and systematically study how predictivity and availability interact to
shape models' feature use. We construct a minimal, explicit generative
framework for synthesizing classification datasets with two latent features
that vary in predictivity and in factors we hypothesize to relate to
availability, and quantify a model's shortcut bias-its over-reliance on the
shortcut (more available, less predictive) feature at the expense of the core
(less available, more predictive) feature. We find that linear models are
relatively unbiased, but introducing a single hidden layer with ReLU or Tanh
units yields a bias. Our empirical findings are consistent with a theoretical
account based on Neural Tangent Kernels. Finally, we study how models used in
practice trade off predictivity and availability in naturalistic datasets,
discovering availability manipulations which increase models' degree of
shortcut bias. Taken together, these findings suggest that the propensity to
learn shortcut features is a fundamental characteristic of deep nonlinear
architectures warranting systematic study given its role in shaping how models
solve tasks.