PICNN: A Pathway towards Interpretable Convolutional Neural Networks
Wengang Guo Jiayi Yang Huilin Yin Qijun Chen Wei Ye
Convolutional Neural Networks (CNNs) have exhibited great performance in
discriminative feature learning for complex visual tasks. Besides
discrimination power, interpretability is another important yet under-explored
property for CNNs. One difficulty in the CNN interpretability is that filters
and image classes are entangled. In this paper, we introduce a novel pathway to
alleviate the entanglement between filters and image classes. The proposed
pathway groups the filters in a late conv-layer of CNN into class-specific
clusters. Clusters and classes are in a one-to-one relationship. Specifically,
we use the Bernoulli sampling to generate the filter-cluster assignment matrix
from a learnable filter-class correspondence matrix. To enable end-to-end
optimization, we develop a novel reparameterization trick for handling the
non-differentiable Bernoulli sampling. We evaluate the effectiveness of our
method on ten widely used network architectures (including nine CNNs and a ViT)
and five benchmark datasets. Experimental results have demonstrated that our
method PICNN (the combination of standard CNNs with our proposed pathway)
exhibits greater interpretability than standard CNNs while achieving higher or
comparable discrimination power.