CoLoRA: Continuous low-rank adaptation for reduced implicit neural
  modeling of parameterized partial differential equations
Jules Berman Benjamin Peherstorfer
This work introduces reduced models based on Continuous Low Rank Adaptation
(CoLoRA) that pre-train neural networks for a given partial differential
equation and then continuously adapt low-rank weights in time to rapidly
predict the evolution of solution fields at new physics parameters and new
initial conditions. The adaptation can be either purely data-driven or via an
equation-driven variational approach that provides Galerkin-optimal
approximations. Because CoLoRA approximates solution fields locally in time,
the rank of the weights can be kept small, which means that only few training
trajectories are required offline so that CoLoRA is well suited for data-scarce
regimes. Predictions with CoLoRA are orders of magnitude faster than with
classical methods and their accuracy and parameter efficiency is higher
compared to other neural network approaches.