An Explainable Deep Framework: Towards Task-Specific Fusion for
  Multi-to-One MRI Synthesis
Luyi Han Tianyu Zhang Yunzhi Huang Haoran Dou Xin Wang Yuan Gao Chunyao Lu Tan Tao Ritse Mann
Multi-sequence MRI is valuable in clinical settings for reliable diagnosis
and treatment prognosis, but some sequences may be unusable or missing for
various reasons. To address this issue, MRI synthesis is a potential solution.
Recent deep learning-based methods have achieved good performance in combining
multiple available sequences for missing sequence synthesis. Despite their
success, these methods lack the ability to quantify the contributions of
different input sequences and estimate the quality of generated images, making
it hard to be practical. Hence, we propose an explainable task-specific
synthesis network, which adapts weights automatically for specific sequence
generation tasks and provides interpretability and reliability from two sides:
(1) visualize the contribution of each input sequence in the fusion stage by a
trainable task-specific weighted average module; (2) highlight the area the
network tried to refine during synthesizing by a task-specific attention
module. We conduct experiments on the BraTS2021 dataset of 1251 subjects, and
results on arbitrary sequence synthesis indicate that the proposed method
achieves better performance than the state-of-the-art methods. Our code is
available at \url{https://github.com/fiy2W/mri_seq2seq}.