Optimized classification with neural ODEs via separability
Antonio Álvarez-López Rafael Orive-Illera Enrique Zuazua
Classification of $N$ points becomes a simultaneous control problem when
viewed through the lens of neural ordinary differential equations (neural
ODEs), which represent the time-continuous limit of residual networks. For the
narrow model, with one neuron per hidden layer, it has been shown that the task
can be achieved using $O(N)$ neurons. In this study, we focus on estimating the
number of neurons required for efficient cluster-based classification,
particularly in the worst-case scenario where points are independently and
uniformly distributed in $[0,1]^d$. Our analysis provides a novel method for
quantifying the probability of requiring fewer than $O(N)$ neurons, emphasizing
the asymptotic behavior as both $d$ and $N$ increase. Additionally, under the
sole assumption that the data are in general position, we propose a new
constructive algorithm that simultaneously classifies clusters of $d$ points
from any initial configuration, effectively reducing the maximal complexity to
$O(N/d)$ neurons.