Generalizing Graph ODE for Learning Complex System Dynamics across
  Environments
Zijie Huang Yizhou Sun Wei Wang
Learning multi-agent system dynamics has been extensively studied for various
real-world applications, such as molecular dynamics in biology. Most of the
existing models are built to learn single system dynamics from observed
historical data and predict the future trajectory. In practice, however, we
might observe multiple systems that are generated across different
environments, which differ in latent exogenous factors such as temperature and
gravity. One simple solution is to learn multiple environment-specific models,
but it fails to exploit the potential commonalities among the dynamics across
environments and offers poor prediction results where per-environment data is
sparse or limited. Here, we present GG-ODE (Generalized Graph Ordinary
Differential Equations), a machine learning framework for learning continuous
multi-agent system dynamics across environments. Our model learns system
dynamics using neural ordinary differential equations (ODE) parameterized by
Graph Neural Networks (GNNs) to capture the continuous interaction among
agents. We achieve the model generalization by assuming the dynamics across
different environments are governed by common physics laws that can be captured
via learning a shared ODE function. The distinct latent exogenous factors
learned for each environment are incorporated into the ODE function to account
for their differences. To improve model performance, we additionally design two
regularization losses to (1) enforce the orthogonality between the learned
initial states and exogenous factors via mutual information minimization; and
(2) reduce the temporal variance of learned exogenous factors within the same
system via contrastive learning. Experiments over various physical simulations
show that our model can accurately predict system dynamics, especially in the
long range, and can generalize well to new systems with few observations.