A Neural RDE approach for continuous-time non-Markovian stochastic
  control problems
Melker Hoglund Emilio Ferrucci Camilo Hernandez Aitor Muguruza Gonzalez Cristopher Salvi Leandro Sanchez-Betancourt Yufei Zhang
We propose a novel framework for solving continuous-time non-Markovian
stochastic control problems by means of neural rough differential equations
(Neural RDEs) introduced in Morrill et al. (2021). Non-Markovianity naturally
arises in control problems due to the time delay effects in the system
coefficients or the driving noises, which leads to optimal control strategies
depending explicitly on the historical trajectories of the system state. By
modelling the control process as the solution of a Neural RDE driven by the
state process, we show that the control-state joint dynamics are governed by an
uncontrolled, augmented Neural RDE, allowing for fast Monte-Carlo estimation of
the value function via trajectories simulation and memory-efficient
backpropagation. We provide theoretical underpinnings for the proposed
algorithmic framework by demonstrating that Neural RDEs serve as universal
approximators for functions of random rough paths. Exhaustive numerical
experiments on non-Markovian stochastic control problems are presented, which
reveal that the proposed framework is time-resolution-invariant and achieves
higher accuracy and better stability in irregular sampling compared to existing
RNN-based approaches.