Multi-task Collaborative Pre-training and Individual-adaptive-tokens
  Fine-tuning: A Unified Framework for Brain Representation Learning
Ning Jiang Gongshu Wang Tianyi Yan
Structural magnetic resonance imaging (sMRI) provides accurate estimates of
the brain's structural organization and learning invariant brain
representations from sMRI is an enduring issue in neuroscience. Previous deep
representation learning models ignore the fact that the brain, as the core of
human cognitive activity, is distinct from other organs whose primary attribute
is anatomy. Therefore, capturing the semantic structure that dominates
interindividual cognitive variability is key to accurately representing the
brain. Given that this high-level semantic information is subtle, distributed,
and interdependently latent in the brain structure, sMRI-based models need to
capture fine-grained details and understand how they relate to the overall
global structure. However, existing models are optimized by simple objectives,
making features collapse into homogeneity and worsening simultaneous
representation of fine-grained information and holistic semantics, causing a
lack of biological plausibility and interpretation of cognition. Here, we
propose MCIAT, a unified framework that combines Multi-task Collaborative
pre-training and Individual-Adaptive-Tokens fine-tuning. Specifically, we first
synthesize restorative learning, age prediction auxiliary learning and
adversarial learning as a joint proxy task for deep semantic representation
learning. Then, a mutual-attention-based token selection method is proposed to
highlight discriminative features. The proposed MCIAT achieves state-of-the-art
diagnosis performance on the ADHD-200 dataset compared with several sMRI-based
approaches and shows superior generalization on the MCIC and OASIS datasets.
Moreover, we studied 12 behavioral tasks and found significant associations
between cognitive functions and MCIAT-established representations, which
verifies the interpretability of our proposed framework.