Algebraic Dynamical Systems in Machine Learning
Iolo Jones Jerry Swan Jeffrey Giansiracusa
We introduce an algebraic analogue of dynamical systems, based on term
rewriting. We show that a recursive function applied to the output of an
iterated rewriting system defines a formal class of models into which all the
main architectures for dynamic machine learning models (including recurrent
neural networks, graph neural networks, and diffusion models) can be embedded.
Considered in category theory, we also show that these algebraic models are a
natural language for describing the compositionality of dynamic models.
Furthermore, we propose that these models provide a template for the
generalisation of the above dynamic models to learning problems on structured
or non-numerical data, including 'hybrid symbolic-numeric' models.