Metric Flows with Neural Networks
James Halverson Fabian Ruehle
We develop a theory of flows in the space of Riemannian metrics induced by
neural network gradient descent. This is motivated in part by recent advances
in approximating Calabi-Yau metrics with neural networks and is enabled by
recent advances in understanding flows in the space of neural networks. We
derive the corresponding metric flow equations, which are governed by a metric
neural tangent kernel, a complicated, non-local object that evolves in time.
However, many architectures admit an infinite-width limit in which the kernel
becomes fixed and the dynamics simplify. Additional assumptions can induce
locality in the flow, which allows for the realization of Perelman's
formulation of Ricci flow that was used to resolve the 3d Poincar\'e
conjecture. We apply these ideas to numerical Calabi-Yau metrics, including a
discussion on the importance of feature learning.