Deep neural networks from the perspective of ergodic theory
Fan Zhang
The design of deep neural networks remains somewhat of an art rather than
precise science. By tentatively adopting ergodic theory considerations on top
of viewing the network as the time evolution of a dynamical system, with each
layer corresponding to a temporal instance, we show that some rules of thumb,
which might otherwise appear mysterious, can be attributed heuristics.