Learning to Explain: A Model-Agnostic Framework for Explaining Black Box
  Models
Oren Barkan Yuval Asher Amit Eshel Yehonatan Elisha Noam Koenigstein
We present Learning to Explain (LTX), a model-agnostic framework designed for
providing post-hoc explanations for vision models. The LTX framework introduces
an "explainer" model that generates explanation maps, highlighting the crucial
regions that justify the predictions made by the model being explained. To
train the explainer, we employ a two-stage process consisting of initial
pretraining followed by per-instance finetuning. During both stages of
training, we utilize a unique configuration where we compare the explained
model's prediction for a masked input with its original prediction for the
unmasked input. This approach enables the use of a novel counterfactual
objective, which aims to anticipate the model's output using masked versions of
the input image. Importantly, the LTX framework is not restricted to a specific
model architecture and can provide explanations for both Transformer-based and
convolutional models. Through our evaluations, we demonstrate that LTX
significantly outperforms the current state-of-the-art in explainability across
various metrics.