Neural Differential Recurrent Neural Network with Adaptive Time Steps
Yixuan Tan Liyan Xie Xiuyuan Cheng
The neural Ordinary Differential Equation (ODE) model has shown success in
learning complex continuous-time processes from observations on discrete time
stamps. In this work, we consider the modeling and forecasting of time series
data that are non-stationary and may have sharp changes like spikes. We propose
an RNN-based model, called RNN-ODE-Adap, that uses a neural ODE to represent
the time development of the hidden states, and we adaptively select time steps
based on the steepness of changes of the data over time so as to train the
model more efficiently for the "spike-like" time series. Theoretically,
RNN-ODE-Adap yields provably a consistent estimation of the intensity function
for the Hawkes-type time series data. We also provide an approximation analysis
of the RNN-ODE model showing the benefit of adaptive steps. The proposed model
is demonstrated to achieve higher prediction accuracy with reduced
computational cost on simulated dynamic system data and point process data and
on a real electrocardiography dataset.