Interpretable Multimodal Learning for Cardiovascular Hemodynamics
  Assessment
Prasun C Tripathi Sina Tabakhi Mohammod N I Suvon Lawrence Schöb Samer Alabed Andrew J Swift Shuo Zhou Haiping Lu
Pulmonary Arterial Wedge Pressure (PAWP) is an essential cardiovascular
hemodynamics marker to detect heart failure. In clinical practice, Right Heart
Catheterization is considered a gold standard for assessing cardiac
hemodynamics while non-invasive methods are often needed to screen high-risk
patients from a large population. In this paper, we propose a multimodal
learning pipeline to predict PAWP marker. We utilize complementary information
from Cardiac Magnetic Resonance Imaging (CMR) scans (short-axis and
four-chamber) and Electronic Health Records (EHRs). We extract spatio-temporal
features from CMR scans using tensor-based learning. We propose a graph
attention network to select important EHR features for prediction, where we
model subjects as graph nodes and feature relationships as graph edges using
the attention mechanism. We design four feature fusion strategies: early,
intermediate, late, and hybrid fusion. With a linear classifier and linear
fusion strategies, our pipeline is interpretable. We validate our pipeline on a
large dataset of $2,641$ subjects from our ASPIRE registry. The comparative
study against state-of-the-art methods confirms the superiority of our
pipeline. The decision curve analysis further validates that our pipeline can
be applied to screen a large population. The code is available at
https://github.com/prasunc/hemodynamics.