Composite Bayesian Optimization In Function Spaces Using NEON -- Neural
  Epistemic Operator Networks
Leonardo Ferreira Guilhoto Paris Perdikaris
Operator learning is a rising field of scientific computing where inputs or
outputs of a machine learning model are functions defined in
infinite-dimensional spaces. In this paper, we introduce NEON (Neural Epistemic
Operator Networks), an architecture for generating predictions with uncertainty
using a single operator network backbone, which presents orders of magnitude
less trainable parameters than deep ensembles of comparable performance. We
showcase the utility of this method for sequential decision-making by examining
the problem of composite Bayesian Optimization (BO), where we aim to optimize a
function $f=g\circ h$, where $h:X\to C(\mathcal{Y},\mathbb{R}^{d_s})$ is an
unknown map which outputs elements of a function space, and $g:
C(\mathcal{Y},\mathbb{R}^{d_s})\to \mathbb{R}$ is a known and cheap-to-compute
functional. By comparing our approach to other state-of-the-art methods on toy
and real world scenarios, we demonstrate that NEON achieves state-of-the-art
performance while requiring orders of magnitude less trainable parameters.