Neural Networks Are Implicit Decision Trees: The Hierarchical Simplicity
  Bias
Zhehang Du
Neural networks exhibit simplicity bias; they rely on simpler features while
ignoring equally predictive but more complex features. In this work, we
introduce a novel approach termed imbalanced label coupling to investigate
scenarios where simple and complex features exhibit different levels of
predictive power. In these cases, complex features still contribute to
predictions. The trained networks make predictions in alignment with the
ascending complexity of input features according to how they correlate with the
label in the training set, irrespective of the underlying predictive power. For
instance, even when simple spurious features distort predictions in CIFAR-10,
most cats are predicted to be dogs, and most trucks are predicted to be
automobiles! This observation provides direct evidence that the neural network
learns core features in the presence of spurious features. We empirically show
that last-layer retraining with target data distribution is effective, yet
insufficient to fully recover core features when spurious features are
perfectly correlated with the target labels in our synthetic dataset. We hope
our research contributes to a deeper understanding of the implicit bias of
neural networks.