Interpretability Benchmark for Evaluating Spatial Misalignment of
  Prototypical Parts Explanations
Mikołaj Sacha Bartosz Jura Dawid Rymarczyk Łukasz Struski Jacek Tabor Bartosz Zieliński
Prototypical parts-based networks are becoming increasingly popular due to
their faithful self-explanations. However, their similarity maps are calculated
in the penultimate network layer. Therefore, the receptive field of the
prototype activation region often depends on parts of the image outside this
region, which can lead to misleading interpretations. We name this undesired
behavior a spatial explanation misalignment and introduce an interpretability
benchmark with a set of dedicated metrics for quantifying this phenomenon. In
addition, we propose a method for misalignment compensation and apply it to
existing state-of-the-art models. We show the expressiveness of our benchmark
and the effectiveness of the proposed compensation methodology through
extensive empirical studies.