Algebraic Complexity and Neurovariety of Linear Convolutional Networks
Vahid Shahverdi
In this paper, we study linear convolutional networks with one-dimensional
filters and arbitrary strides. The neuromanifold of such a network is a
semialgebraic set, represented by a space of polynomials admitting specific
factorizations. Introducing a recursive algorithm, we generate polynomial
equations whose common zero locus corresponds to the Zariski closure of the
corresponding neuromanifold. Furthermore, we explore the algebraic complexity
of training these networks employing tools from metric algebraic geometry. Our
findings reveal that the number of all complex critical points in the
optimization of such a network is equal to the generic Euclidean distance
degree of a Segre variety. Notably, this count significantly surpasses the
number of critical points encountered in the training of a fully connected
linear network with the same number of parameters.