A singular Riemannian Geometry Approach to Deep Neural Networks III.
  Piecewise Differentiable Layers and Random Walks on $n$-dimensional Classes
Alessandro Benfenati Alessio Marta
Neural networks are playing a crucial role in everyday life, with the most
modern generative models able to achieve impressive results. Nonetheless, their
functioning is still not very clear, and several strategies have been adopted
to study how and why these model reach their outputs. A common approach is to
consider the data in an Euclidean settings: recent years has witnessed instead
a shift from this paradigm, moving thus to more general framework, namely
Riemannian Geometry. Two recent works introduced a geometric framework to study
neural networks making use of singular Riemannian metrics. In this paper we
extend these results to convolutional, residual and recursive neural networks,
studying also the case of non-differentiable activation functions, such as
ReLU. We illustrate our findings with some numerical experiments on
classification of images and thermodynamic problems.