Neural Oscillators are Universal
Samuel Lanthaler T. Konstantin Rusch Siddhartha Mishra
Coupled oscillators are being increasingly used as the basis of machine
learning (ML) architectures, for instance in sequence modeling, graph
representation learning and in physical neural networks that are used in analog
ML devices. We introduce an abstract class of neural oscillators that
encompasses these architectures and prove that neural oscillators are
universal, i.e, they can approximate any continuous and casual operator mapping
between time-varying functions, to desired accuracy. This universality result
provides theoretical justification for the use of oscillator based ML systems.
The proof builds on a fundamental result of independent interest, which shows
that a combination of forced harmonic oscillators with a nonlinear read-out
suffices to approximate the underlying operators.