Foundational Inference Models for Dynamical Systems
Patrick Seifner Kostadin Cvejoski Ramses J. Sanchez
Ordinary differential equations (ODEs) underlie dynamical systems which serve
as models for a vast number of natural and social phenomena. Yet inferring the
ODE that best describes a set of noisy observations on one such phenomenon can
be remarkably challenging, and the models available to achieve it tend to be
highly specialized and complex too. In this work we propose a novel supervised
learning framework for zero-shot inference of ODEs from noisy data. We first
generate large datasets of one-dimensional ODEs, by sampling distributions over
the space of initial conditions, and the space of vector fields defining them.
We then learn neural maps between noisy observations on the solutions of these
equations, and their corresponding initial condition and vector fields. The
resulting models, which we call foundational inference models (FIM), can be (i)
copied and matched along the time dimension to increase their resolution; and
(ii) copied and composed to build inference models of any dimensionality,
without the need of any finetuning. We use FIM to model both ground-truth
dynamical systems of different dimensionalities and empirical time series data
in a zero-shot fashion, and outperform state-of-the-art models which are
finetuned to these systems. Our (pretrained) FIMs are available online