Geometry of Polynomial Neural Networks
Kaie Kubjas Jiayi Li Maximilian Wiesmann
We study the expressivity and learning process for polynomial neural networks
(PNNs) with monomial activation functions. The weights of the network
parametrize the neuromanifold. In this paper, we study certain neuromanifolds
using tools from algebraic geometry: we give explicit descriptions as
semialgebraic sets and characterize their Zariski closures, called
neurovarieties. We study their dimension and associate an algebraic degree, the
learning degree, to the neurovariety. The dimension serves as a geometric
measure for the expressivity of the network, the learning degree is a measure
for the complexity of training the network and provides upper bounds on the
number of learnable functions. These theoretical results are accompanied with
experiments.