Emergent Language Symbolic Autoencoder (ELSA) with Weak Supervision to
  Model Hierarchical Brain Networks
Ammar Ahmed Pallikonda Latheef Alberto Santamaria-Pang Craig K Jones Haris I Sair
Brain networks display a hierarchical organization, a complexity that poses a
challenge for existing deep learning models, often structured as flat
classifiers, leading to difficulties in interpretability and the 'black box'
issue. To bridge this gap, we propose a novel architecture: a symbolic
autoencoder informed by weak supervision and an Emergent Language (EL)
framework. This model moves beyond traditional flat classifiers by producing
hierarchical clusters and corresponding imagery, subsequently represented
through symbolic sentences to improve the clinical interpretability of
hierarchically organized data such as intrinsic brain networks, which can be
characterized using resting-state fMRI images. Our innovation includes a
generalized hierarchical loss function designed to ensure that both sentences
and images accurately reflect the hierarchical structure of functional brain
networks. This enables us to model functional brain networks from a broader
perspective down to more granular details. Furthermore, we introduce a
quantitative method to assess the hierarchical consistency of these symbolic
representations. Our qualitative analyses show that our model successfully
generates hierarchically organized, clinically interpretable images, a finding
supported by our quantitative evaluations. We find that our best performing
loss function leads to a hierarchical consistency of over 97% when identifying
images corresponding to brain networks. This approach not only advances the
interpretability of deep learning models in neuroimaging analysis but also
represents a significant step towards modeling the intricate hierarchical
nature of brain networks.