Generative Perturbation Analysis for Probabilistic Black-Box Anomaly
  Attribution
Tsuyoshi Idé Naoki Abe
We address the task of probabilistic anomaly attribution in the black-box
regression setting, where the goal is to compute the probability distribution
of the attribution score of each input variable, given an observed anomaly. The
training dataset is assumed to be unavailable. This task differs from the
standard XAI (explainable AI) scenario, since we wish to explain the anomalous
deviation from a black-box prediction rather than the black-box model itself.
  We begin by showing that mainstream model-agnostic explanation methods, such
as the Shapley values, are not suitable for this task because of their
``deviation-agnostic property.'' We then propose a novel framework for
probabilistic anomaly attribution that allows us to not only compute
attribution scores as the predictive mean but also quantify the uncertainty of
those scores. This is done by considering a generative process for
perturbations that counter-factually bring the observed anomalous observation
back to normalcy. We introduce a variational Bayes algorithm for deriving the
distributions of per variable attribution scores. To the best of our knowledge,
this is the first probabilistic anomaly attribution framework that is free from
being deviation-agnostic.