LSOR: Longitudinally-Consistent Self-Organized Representation Learning
Jiahong Ouyang Qingyu Zhao Ehsan Adeli Wei Peng Greg Zaharchuk Kilian M. Pohl
Interpretability is a key issue when applying deep learning models to
longitudinal brain MRIs. One way to address this issue is by visualizing the
high-dimensional latent spaces generated by deep learning via self-organizing
maps (SOM). SOM separates the latent space into clusters and then maps the
cluster centers to a discrete (typically 2D) grid preserving the
high-dimensional relationship between clusters. However, learning SOM in a
high-dimensional latent space tends to be unstable, especially in a
self-supervision setting. Furthermore, the learned SOM grid does not
necessarily capture clinically interesting information, such as brain age. To
resolve these issues, we propose the first self-supervised SOM approach that
derives a high-dimensional, interpretable representation stratified by brain
age solely based on longitudinal brain MRIs (i.e., without demographic or
cognitive information). Called Longitudinally-consistent Self-Organized
Representation learning (LSOR), the method is stable during training as it
relies on soft clustering (vs. the hard cluster assignments used by existing
SOM). Furthermore, our approach generates a latent space stratified according
to brain age by aligning trajectories inferred from longitudinal MRIs to the
reference vector associated with the corresponding SOM cluster. When applied to
longitudinal MRIs of the Alzheimer's Disease Neuroimaging Initiative (ADNI,
N=632), LSOR generates an interpretable latent space and achieves comparable or
higher accuracy than the state-of-the-art representations with respect to the
downstream tasks of classification (static vs. progressive mild cognitive
impairment) and regression (determining ADAS-Cog score of all subjects). The
code is available at
https://github.com/ouyangjiahong/longitudinal-som-single-modality.