Towards credible visual model interpretation with path attribution
Naveed Akhtar Muhammad A. A. K. Jalwana
Originally inspired by game-theory, path attribution framework stands out
among the post-hoc model interpretation tools due to its axiomatic nature.
However, recent developments show that this framework can still suffer from
counter-intuitive results. Moreover, specifically for deep visual models, the
existing path-based methods also fall short on conforming to the original
intuitions that are the basis of the claimed axiomatic properties of this
framework. We address these problems with a systematic investigation, and
pinpoint the conditions in which the counter-intuitive results can be avoided
for deep visual model interpretation with the path attribution strategy. We
also devise a scheme to preclude the conditions in which visual model
interpretation can invalidate the axiomatic properties of path attribution.
These insights are combined into a method that enables reliable visual model
interpretation. Our findings are establish empirically with multiple datasets,
models and evaluation metrics. Extensive experiments show a consistent
performance gain of our method over the baselines.