The Boundaries of Verifiable Accuracy, Robustness, and Generalisation in
  Deep Learning
Alexander Bastounis Alexander N. Gorban Anders C. Hansen Desmond J. Higham Danil Prokhorov Oliver Sutton Ivan Y. Tyukin Qinghua Zhou
In this work, we assess the theoretical limitations of determining guaranteed
stability and accuracy of neural networks in classification tasks. We consider
classical distribution-agnostic framework and algorithms minimising empirical
risks and potentially subjected to some weights regularisation. We show that
there is a large family of tasks for which computing and verifying ideal stable
and accurate neural networks in the above settings is extremely challenging, if
at all possible, even when such ideal solutions exist within the given class of
neural architectures.