Learning Stochastic Dynamical Systems as an Implicit Regularization with
  Graph Neural Networks
Jin Guo Ting Gao Yufu Lan Peng Zhang Sikun Yang Jinqiao Duan
Stochastic Gumbel graph networks are proposed to learn high-dimensional time
series, where the observed dimensions are often spatially correlated. To that
end, the observed randomness and spatial-correlations are captured by learning
the drift and diffusion terms of the stochastic differential equation with a
Gumble matrix embedding, respectively. In particular, this novel framework
enables us to investigate the implicit regularization effect of the noise terms
in S-GGNs. We provide a theoretical guarantee for the proposed S-GGNs by
deriving the difference between the two corresponding loss functions in a small
neighborhood of weight. Then, we employ Kuramoto's model to generate data for
comparing the spectral density from the Hessian Matrix of the two loss
functions. Experimental results on real-world data, demonstrate that S-GGNs
exhibit superior convergence, robustness, and generalization, compared with
state-of-the-arts.