Mathematics of Neural Networks (Lecture Notes Graduate Course)
Bart M. N. Smets
These are the lecture notes that accompanied the course of the same name that
I taught at the Eindhoven University of Technology from 2021 to 2023. The
course is intended as an introduction to neural networks for mathematics
students at the graduate level and aims to make mathematics students interested
in further researching neural networks. It consists of two parts: first a
general introduction to deep learning that focuses on introducing the field in
a formal mathematical way. The second part provides an introduction to the
theory of Lie groups and homogeneous spaces and how it can be applied to design
neural networks with desirable geometric equivariances. The lecture notes were
made to be as self-contained as possible so as to accessible for any student
with a moderate mathematics background. The course also included coding
tutorials and assignments in the form of a set of Jupyter notebooks that are
publicly available at
https://gitlab.com/bsmetsjr/mathematics_of_neural_networks.