A Knowledge Distillation Approach for Sepsis Outcome Prediction from
  Multivariate Clinical Time Series
Anna Wong Shu Ge Nassim Oufattole Adam Dejl Megan Su Ardavan Saeedi Li-wei H. Lehman
Sepsis is a life-threatening condition triggered by an extreme infection
response. Our objective is to forecast sepsis patient outcomes using their
medical history and treatments, while learning interpretable state
representations to assess patients' risks in developing various adverse
outcomes. While neural networks excel in outcome prediction, their limited
interpretability remains a key issue. In this work, we use knowledge
distillation via constrained variational inference to distill the knowledge of
a powerful "teacher" neural network model with high predictive power to train a
"student" latent variable model to learn interpretable hidden state
representations to achieve high predictive performance for sepsis outcome
prediction. Using real-world data from the MIMIC-IV database, we trained an
LSTM as the "teacher" model to predict mortality for sepsis patients, given
information about their recent history of vital signs, lab values and
treatments. For our student model, we use an autoregressive hidden Markov model
(AR-HMM) to learn interpretable hidden states from patients' clinical time
series, and use the posterior distribution of the learned state representations
to predict various downstream outcomes, including hospital mortality, pulmonary
edema, need for diuretics, dialysis, and mechanical ventilation. Our results
show that our approach successfully incorporates the constraint to achieve high
predictive power similar to the teacher model, while maintaining the generative
performance.