Log Neural Controlled Differential Equations: The Lie Brackets Make a
  Difference
Benjamin Walker Andrew D. McLeod Tiexin Qin Yichuan Cheng Haoliang Li Terry Lyons
The vector field of a controlled differential equation (CDE) describes the
relationship between a control path and the evolution of a solution path.
Neural CDEs (NCDEs) treat time series data as observations from a control path,
parameterise a CDE's vector field using a neural network, and use the solution
path as a continuously evolving hidden state. As their formulation makes them
robust to irregular sampling rates, NCDEs are a powerful approach for modelling
real-world data. Building on neural rough differential equations (NRDEs), we
introduce Log-NCDEs, a novel and effective method for training NCDEs. The core
component of Log-NCDEs is the Log-ODE method, a tool from the study of rough
paths for approximating a CDE's solution. On a range of multivariate time
series classification benchmarks, Log-NCDEs are shown to achieve a higher
average test set accuracy than NCDEs, NRDEs, and two state-of-the-art models,
S5 and the linear recurrent unit.