A note on the adjoint method for neural ordinary differential equation
  network
Pipi Hu
Perturbation and operator adjoint method are used to give the right adjoint
form rigourously. From the derivation, we can have following results: 1) The
loss gradient is not an ODE, it is an integral and we shows the reason; 2) The
traditional adjoint form is not equivalent with the back propagation results.
3) The adjoint operator analysis shows that if and only if the discrete adjoint
has the same scheme with the discrete neural ODE, the adjoint form would give
the same results as BP does.