On the Importance of Step-wise Embeddings for Heterogeneous Clinical
  Time-Series
Rita Kuznetsova Alizée Pace Manuel Burger Hugo Yèche Gunnar Rätsch
Recent advances in deep learning architectures for sequence modeling have not
fully transferred to tasks handling time-series from electronic health records.
In particular, in problems related to the Intensive Care Unit (ICU), the
state-of-the-art remains to tackle sequence classification in a tabular manner
with tree-based methods. Recent findings in deep learning for tabular data are
now surpassing these classical methods by better handling the severe
heterogeneity of data input features. Given the similar level of feature
heterogeneity exhibited by ICU time-series and motivated by these findings, we
explore these novel methods' impact on clinical sequence modeling tasks. By
jointly using such advances in deep learning for tabular data, our primary
objective is to underscore the importance of step-wise embeddings in
time-series modeling, which remain unexplored in machine learning methods for
clinical data. On a variety of clinically relevant tasks from two large-scale
ICU datasets, MIMIC-III and HiRID, our work provides an exhaustive analysis of
state-of-the-art methods for tabular time-series as time-step embedding models,
showing overall performance improvement. In particular, we evidence the
importance of feature grouping in clinical time-series, with significant
performance gains when considering features within predefined semantic groups
in the step-wise embedding module.