HOLMES: HOLonym-MEronym based Semantic inspection for Convolutional
  Image Classifiers
Francesco Dibitonto Fabio Garcea André Panisson Alan Perotti Lia Morra
Convolutional Neural Networks (CNNs) are nowadays the model of choice in
Computer Vision, thanks to their ability to automatize the feature extraction
process in visual tasks. However, the knowledge acquired during training is
fully subsymbolic, and hence difficult to understand and explain to end users.
In this paper, we propose a new technique called HOLMES (HOLonym-MEronym based
Semantic inspection) that decomposes a label into a set of related concepts,
and provides component-level explanations for an image classification model.
Specifically, HOLMES leverages ontologies, web scraping and transfer learning
to automatically construct meronym (parts)-based detectors for a given holonym
(class). Then, it produces heatmaps at the meronym level and finally, by
probing the holonym CNN with occluded images, it highlights the importance of
each part on the classification output. Compared to state-of-the-art saliency
methods, HOLMES takes a step further and provides information about both where
and what the holonym CNN is looking at, without relying on densely annotated
datasets and without forcing concepts to be associated to single computational
units. Extensive experimental evaluation on different categories of objects
(animals, tools and vehicles) shows the feasibility of our approach. On
average, HOLMES explanations include at least two meronyms, and the ablation of
a single meronym roughly halves the holonym model confidence. The resulting
heatmaps were quantitatively evaluated using the
deletion/insertion/preservation curves. All metrics were comparable to those
achieved by GradCAM, while offering the advantage of further decomposing the
heatmap in human-understandable concepts, thus highlighting both the relevance
of meronyms to object classification, as well as HOLMES ability to capture it.
The code is available at https://github.com/FrancesC0de/HOLMES.