Brain Structure-Function Fusing Representation Learning using
  Adversarial Decomposed-VAE for Analyzing MCI
Qiankun Zuo Baiying Lei Ning Zhong Yi Pan Shuqiang Wang
Integrating the brain structural and functional connectivity features is of
great significance in both exploring brain science and analyzing cognitive
impairment clinically. However, it remains a challenge to effectively fuse
structural and functional features in exploring the brain network. In this
paper, a novel brain structure-function fusing-representation learning (BSFL)
model is proposed to effectively learn fused representation from diffusion
tensor imaging (DTI) and resting-state functional magnetic resonance imaging
(fMRI) for mild cognitive impairment (MCI) analysis. Specifically, the
decomposition-fusion framework is developed to first decompose the feature
space into the union of the uniform and the unique spaces for each modality,
and then adaptively fuse the decomposed features to learn MCI-related
representation. Moreover, a knowledge-aware transformer module is designed to
automatically capture local and global connectivity features throughout the
brain. Also, a uniform-unique contrastive loss is further devised to make the
decomposition more effective and enhance the complementarity of structural and
functional features. The extensive experiments demonstrate that the proposed
model achieves better performance than other competitive methods in predicting
and analyzing MCI. More importantly, the proposed model could be a potential
tool for reconstructing unified brain networks and predicting abnormal
connections during the degenerative processes in MCI.