Symmetry-regularized neural ordinary differential equations
Wenbo Hao
Neural Ordinary Differential Equations (Neural ODEs) is a class of deep
neural network models that interpret the hidden state dynamics of neural
networks as an ordinary differential equation, thereby capable of capturing
system dynamics in a continuous time framework. In this work, I integrate
symmetry regularization into Neural ODEs. In particular, I use continuous Lie
symmetry of ODEs and PDEs associated with the model to derive conservation laws
and add them to the loss function, making it physics-informed. This
incorporation of inherent structural properties into the loss function could
significantly improve robustness and stability of the model during training. To
illustrate this method, I employ a toy model that utilizes a cosine rate of
change in the hidden state, showcasing the process of identifying Lie
symmetries, deriving conservation laws, and constructing a new loss function.