Epistemic Modeling Uncertainty of Rapid Neural Network Ensembles for
  Adaptive Learning
Atticus Beachy Harok Bae Jose Camberos Ramana Grandhi
Emulator embedded neural networks, which are a type of physics informed
neural network, leverage multi-fidelity data sources for efficient design
exploration of aerospace engineering systems. Multiple realizations of the
neural network models are trained with different random initializations. The
ensemble of model realizations is used to assess epistemic modeling uncertainty
caused due to lack of training samples. This uncertainty estimation is crucial
information for successful goal-oriented adaptive learning in an aerospace
system design exploration. However, the costs of training the ensemble models
often become prohibitive and pose a computational challenge, especially when
the models are not trained in parallel during adaptive learning. In this work,
a new type of emulator embedded neural network is presented using the rapid
neural network paradigm. Unlike the conventional neural network training that
optimizes the weights and biases of all the network layers by using
gradient-based backpropagation, rapid neural network training adjusts only the
last layer connection weights by applying a linear regression technique. It is
found that the proposed emulator embedded neural network trains
near-instantaneously, typically without loss of prediction accuracy. The
proposed method is demonstrated on multiple analytical examples, as well as an
aerospace flight parameter study of a generic hypersonic vehicle.