Dynamic Model Agnostic Reliability Evaluation of Machine-Learning
  Methods Integrated in Instrumentation & Control Systems
Edward Chen Han Bao Nam Dinh
In recent years, the field of data-driven neural network-based machine
learning (ML) algorithms has grown significantly and spurred research in its
applicability to instrumentation and control systems. While they are promising
in operational contexts, the trustworthiness of such algorithms is not
adequately assessed. Failures of ML-integrated systems are poorly understood;
the lack of comprehensive risk modeling can degrade the trustworthiness of
these systems. In recent reports by the National Institute for Standards and
Technology, trustworthiness in ML is a critical barrier to adoption and will
play a vital role in intelligent systems' safe and accountable operation. Thus,
in this work, we demonstrate a real-time model-agnostic method to evaluate the
relative reliability of ML predictions by incorporating out-of-distribution
detection on the training dataset. It is well documented that ML algorithms
excel at interpolation (or near-interpolation) tasks but significantly degrade
at extrapolation. This occurs when new samples are "far" from training samples.
The method, referred to as the Laplacian distributed decay for reliability
(LADDR), determines the difference between the operational and training
datasets, which is used to calculate a prediction's relative reliability. LADDR
is demonstrated on a feedforward neural network-based model used to predict
safety significant factors during different loss-of-flow transients. LADDR is
intended as a "data supervisor" and determines the appropriateness of
well-trained ML models in the context of operational conditions. Ultimately,
LADDR illustrates how training data can be used as evidence to support the
trustworthiness of ML predictions when utilized for conventional interpolation
tasks.