This Looks Like Those: Illuminating Prototypical Concepts Using Multiple
  Visualizations
Chiyu Ma Brandon Zhao Chaofan Chen Cynthia Rudin
We present ProtoConcepts, a method for interpretable image classification
combining deep learning and case-based reasoning using prototypical parts.
Existing work in prototype-based image classification uses a ``this looks like
that'' reasoning process, which dissects a test image by finding prototypical
parts and combining evidence from these prototypes to make a final
classification. However, all of the existing prototypical part-based image
classifiers provide only one-to-one comparisons, where a single training image
patch serves as a prototype to compare with a part of our test image. With
these single-image comparisons, it can often be difficult to identify the
underlying concept being compared (e.g., ``is it comparing the color or the
shape?''). Our proposed method modifies the architecture of prototype-based
networks to instead learn prototypical concepts which are visualized using
multiple image patches. Having multiple visualizations of the same prototype
allows us to more easily identify the concept captured by that prototype (e.g.,
``the test image and the related training patches are all the same shade of
blue''), and allows our model to create richer, more interpretable visual
explanations. Our experiments show that our ``this looks like those'' reasoning
process can be applied as a modification to a wide range of existing
prototypical image classification networks while achieving comparable accuracy
on benchmark datasets.