Towards a fuller understanding of neurons with Clustered Compositional
  Explanations
Biagio La Rosa Leilani H. Gilpin Roberto Capobianco
Compositional Explanations is a method for identifying logical formulas of
concepts that approximate the neurons' behavior. However, these explanations
are linked to the small spectrum of neuron activations (i.e., the highest ones)
used to check the alignment, thus lacking completeness. In this paper, we
propose a generalization, called Clustered Compositional Explanations, that
combines Compositional Explanations with clustering and a novel search
heuristic to approximate a broader spectrum of the neurons' behavior. We define
and address the problems connected to the application of these methods to
multiple ranges of activations, analyze the insights retrievable by using our
algorithm, and propose desiderata qualities that can be used to study the
explanations returned by different algorithms.