Universal Approximation and the Topological Neural Network
Michael A. Kouritzin Daniel Richard
A topological neural network (TNN), which takes data from a Tychonoff
topological space instead of the usual finite dimensional space, is introduced.
As a consequence, a distributional neural network (DNN) that takes Borel
measures as data is also introduced. Combined these new neural networks
facilitate things like recognizing long range dependence, heavy tails and other
properties in stochastic process paths or like acting on belief states produced
by particle filtering or hidden Markov model algorithms. The veracity of the
TNN and DNN are then established herein by a strong universal approximation
theorem for Tychonoff spaces and its corollary for spaces of measures. These
theorems show that neural networks can arbitrarily approximate uniformly
continuous functions (with respect to the sup metric) associated with a unique
uniformity. We also provide some discussion showing that neural networks on
positive-finite measures are a generalization of the recent deep learning
notion of deep sets.