Listenable Maps for Audio Classifiers
Francesco Paissan Mirco Ravanelli Cem Subakan
Despite the impressive performance of deep learning models across diverse
tasks, their complexity poses challenges for interpretation. This challenge is
particularly evident for audio signals, where conveying interpretations becomes
inherently difficult. To address this issue, we introduce Listenable Maps for
Audio Classifiers (L-MAC), a posthoc interpretation method that generates
faithful and listenable interpretations. L-MAC utilizes a decoder on top of a
pretrained classifier to generate binary masks that highlight relevant portions
of the input audio. We train the decoder with a special loss that maximizes the
confidence of the classifier decision on the masked-in portion of the audio
while minimizing the probability of model output for the masked-out portion.
Quantitative evaluations on both in-domain and out-of-domain data demonstrate
that L-MAC consistently produces more faithful interpretations than several
gradient and masking-based methodologies. Furthermore, a user study confirms
that, on average, users prefer the interpretations generated by the proposed
technique.