Efficient geometric Markov chain Monte Carlo for nonlinear Bayesian
  inversion enabled by derivative-informed neural operators
Lianghao Cao Thomas O'Leary-Roseberry Omar Ghattas
We propose an operator learning approach to accelerate geometric Markov chain
Monte Carlo (MCMC) for solving infinite-dimensional nonlinear Bayesian inverse
problems. While geometric MCMC employs high-quality proposals that adapt to
posterior local geometry, it requires computing local gradient and Hessian
information of the log-likelihood, incurring a high cost when the
parameter-to-observable (PtO) map is defined through expensive model
simulations. We consider a delayed-acceptance geometric MCMC method driven by a
neural operator surrogate of the PtO map, where the proposal is designed to
exploit fast surrogate approximations of the log-likelihood and,
simultaneously, its gradient and Hessian. To achieve a substantial speedup, the
surrogate needs to be accurate in predicting both the observable and its
parametric derivative (the derivative of the observable with respect to the
parameter). Training such a surrogate via conventional operator learning using
input--output samples often demands a prohibitively large number of model
simulations. In this work, we present an extension of derivative-informed
operator learning [O'Leary-Roseberry et al., J. Comput. Phys., 496 (2024)]
using input--output--derivative training samples. Such a learning method leads
to derivative-informed neural operator (DINO) surrogates that accurately
predict the observable and its parametric derivative at a significantly lower
training cost than the conventional method. Cost and error analysis for reduced
basis DINO surrogates are provided. Numerical studies on PDE-constrained
Bayesian inversion demonstrate that DINO-driven MCMC generates effective
posterior samples 3--9 times faster than geometric MCMC and 60--97 times faster
than prior geometry-based MCMC. Furthermore, the training cost of DINO
surrogates breaks even after collecting merely 10--25 effective posterior
samples compared to geometric MCMC.