Inferring Data Preconditions from Deep Learning Models for Trustworthy
  Prediction in Deployment
Shibbir Ahmed Hongyang Gao Hridesh Rajan
Deep learning models are trained with certain assumptions about the data
during the development stage and then used for prediction in the deployment
stage. It is important to reason about the trustworthiness of the model's
predictions with unseen data during deployment. Existing methods for specifying
and verifying traditional software are insufficient for this task, as they
cannot handle the complexity of DNN model architecture and expected outcomes.
In this work, we propose a novel technique that uses rules derived from neural
network computations to infer data preconditions for a DNN model to determine
the trustworthiness of its predictions. Our approach, DeepInfer involves
introducing a novel abstraction for a trained DNN model that enables weakest
precondition reasoning using Dijkstra's Predicate Transformer Semantics. By
deriving rules over the inductive type of neural network abstract
representation, we can overcome the matrix dimensionality issues that arise
from the backward non-linear computation from the output layer to the input
layer. We utilize the weakest precondition computation using rules of each kind
of activation function to compute layer-wise precondition from the given
postcondition on the final output of a deep neural network. We extensively
evaluated DeepInfer on 29 real-world DNN models using four different datasets
collected from five different sources and demonstrated the utility,
effectiveness, and performance improvement over closely related work. DeepInfer
efficiently detects correct and incorrect predictions of high-accuracy models
with high recall (0.98) and high F-1 score (0.84) and has significantly
improved over prior technique, SelfChecker. The average runtime overhead of
DeepInfer is low, 0.22 sec for all unseen datasets. We also compared runtime
overhead using the same hardware settings and found that DeepInfer is 3.27
times faster than SelfChecker.