Leveraging viscous Hamilton-Jacobi PDEs for uncertainty quantification
  in scientific machine learning
Zongren Zou Tingwei Meng Paula Chen Jérôme Darbon George Em Karniadakis
Uncertainty quantification (UQ) in scientific machine learning (SciML)
combines the powerful predictive power of SciML with methods for quantifying
the reliability of the learned models. However, two major challenges remain:
limited interpretability and expensive training procedures. We provide a new
interpretation for UQ problems by establishing a new theoretical connection
between some Bayesian inference problems arising in SciML and viscous
Hamilton-Jacobi partial differential equations (HJ PDEs). Namely, we show that
the posterior mean and covariance can be recovered from the spatial gradient
and Hessian of the solution to a viscous HJ PDE. As a first exploration of this
connection, we specialize to Bayesian inference problems with linear models,
Gaussian likelihoods, and Gaussian priors. In this case, the associated viscous
HJ PDEs can be solved using Riccati ODEs, and we develop a new Riccati-based
methodology that provides computational advantages when continuously updating
the model predictions. Specifically, our Riccati-based approach can efficiently
add or remove data points to the training set invariant to the order of the
data and continuously tune hyperparameters. Moreover, neither update requires
retraining on or access to previously incorporated data. We provide several
examples from SciML involving noisy data and \textit{epistemic uncertainty} to
illustrate the potential advantages of our approach. In particular, this
approach's amenability to data streaming applications demonstrates its
potential for real-time inferences, which, in turn, allows for applications in
which the predicted uncertainty is used to dynamically alter the learning
process.