MM-SurvNet: Deep Learning-Based Survival Risk Stratification in Breast
  Cancer Through Multimodal Data Fusion
Raktim Kumar Mondol Ewan K. A. Millar Arcot Sowmya Erik Meijering
Survival risk stratification is an important step in clinical decision making
for breast cancer management. We propose a novel deep learning approach for
this purpose by integrating histopathological imaging, genetic and clinical
data. It employs vision transformers, specifically the MaxViT model, for image
feature extraction, and self-attention to capture intricate image relationships
at the patient level. A dual cross-attention mechanism fuses these features
with genetic data, while clinical data is incorporated at the final layer to
enhance predictive accuracy. Experiments on the public TCGA-BRCA dataset show
that our model, trained using the negative log likelihood loss function, can
achieve superior performance with a mean C-index of 0.64, surpassing existing
methods. This advancement facilitates tailored treatment strategies,
potentially leading to improved patient outcomes.