Exploring higher-order neural network node interactions with total
  correlation
Thomas Kerby Teresa White Kevin Moon
In domains such as ecological systems, collaborations, and the human brain
the variables interact in complex ways. Yet accurately characterizing
higher-order variable interactions (HOIs) is a difficult problem that is
further exacerbated when the HOIs change across the data. To solve this problem
we propose a new method called Local Correlation Explanation (CorEx) to capture
HOIs at a local scale by first clustering data points based on their proximity
on the data manifold. We then use a multivariate version of the mutual
information called the total correlation, to construct a latent factor
representation of the data within each cluster to learn the local HOIs. We use
Local CorEx to explore HOIs in synthetic and real world data to extract hidden
insights about the data structure. Lastly, we demonstrate Local CorEx's
suitability to explore and interpret the inner workings of trained neural
networks.