Be Persistent: Towards a Unified Solution for Mitigating Shortcuts in
  Deep Learning
Hadi M. Dolatabadi Sarah M. Erfani Christopher Leckie
Deep neural networks (DNNs) are vulnerable to shortcut learning: rather than
learning the intended task, they tend to draw inconclusive relationships
between their inputs and outputs. Shortcut learning is ubiquitous among many
failure cases of neural networks, and traces of this phenomenon can be seen in
their generalizability issues, domain shift, adversarial vulnerability, and
even bias towards majority groups. In this paper, we argue that this
commonality in the cause of various DNN issues creates a significant
opportunity that should be leveraged to find a unified solution for shortcut
learning. To this end, we outline the recent advances in topological data
analysis~(TDA), and persistent homology~(PH) in particular, to sketch a unified
roadmap for detecting shortcuts in deep learning. We demonstrate our arguments
by investigating the topological features of computational graphs in DNNs using
two cases of unlearnable examples and bias in decision-making as our test
studies. Our analysis of these two failure cases of DNNs reveals that finding a
unified solution for shortcut learning in DNNs is not out of reach, and TDA can
play a significant role in forming such a framework.