DiffHybrid-UQ: Uncertainty Quantification for Differentiable Hybrid
  Neural Modeling
Deepak Akhare Tengfei Luo Jian-Xun Wang
The hybrid neural differentiable models mark a significant advancement in the
field of scientific machine learning. These models, integrating numerical
representations of known physics into deep neural networks, offer enhanced
predictive capabilities and show great potential for data-driven modeling of
complex physical systems. However, a critical and yet unaddressed challenge
lies in the quantification of inherent uncertainties stemming from multiple
sources. Addressing this gap, we introduce a novel method, DiffHybrid-UQ, for
effective and efficient uncertainty propagation and estimation in hybrid neural
differentiable models, leveraging the strengths of deep ensemble Bayesian
learning and nonlinear transformations. Specifically, our approach effectively
discerns and quantifies both aleatoric uncertainties, arising from data noise,
and epistemic uncertainties, resulting from model-form discrepancies and data
sparsity. This is achieved within a Bayesian model averaging framework, where
aleatoric uncertainties are modeled through hybrid neural models. The unscented
transformation plays a pivotal role in enabling the flow of these uncertainties
through the nonlinear functions within the hybrid model. In contrast, epistemic
uncertainties are estimated using an ensemble of stochastic gradient descent
(SGD) trajectories. This approach offers a practical approximation to the
posterior distribution of both the network parameters and the physical
parameters. Notably, the DiffHybrid-UQ framework is designed for simplicity in
implementation and high scalability, making it suitable for parallel computing
environments. The merits of the proposed method have been demonstrated through
problems governed by both ordinary and partial differentiable equations.