Diffexplainer: Towards Cross-modal Global Explanations with Diffusion
  Models
Matteo Pennisi Giovanni Bellitto Simone Palazzo Mubarak Shah Concetto Spampinato
We present DiffExplainer, a novel framework that, leveraging language-vision
models, enables multimodal global explainability. DiffExplainer employs
diffusion models conditioned on optimized text prompts, synthesizing images
that maximize class outputs and hidden features of a classifier, thus providing
a visual tool for explaining decisions. Moreover, the analysis of generated
visual descriptions allows for automatic identification of biases and spurious
features, as opposed to traditional methods that often rely on manual
intervention. The cross-modal transferability of language-vision models also
enables the possibility to describe decisions in a more human-interpretable
way, i.e., through text. We conduct comprehensive experiments, which include an
extensive user study, demonstrating the effectiveness of DiffExplainer on 1)
the generation of high-quality images explaining model decisions, surpassing
existing activation maximization methods, and 2) the automated identification
of biases and spurious features.