Dynamic Local Attention with Hierarchical Patching for Irregular
  Clinical Time Series
Xingyu Chen Xiaochen Zheng Amina Mollaysa Manuel Schürch Ahmed Allam Michael Krauthammer
Irregular multivariate time series data is prevalent in the clinical and
healthcare domains. It is characterized by time-wise and feature-wise
irregularities, making it challenging for machine learning methods to work
with. To solve this, we introduce a new model architecture composed of two
modules: (1) DLA, a Dynamic Local Attention mechanism that uses learnable
queries and feature-specific local windows when computing the self-attention
operation. This results in aggregating irregular time steps raw input within
each window to a harmonized regular latent space representation while taking
into account the different features' sampling rates. (2) A hierarchical MLP
mixer that processes the output of DLA through multi-scale patching to leverage
information at various scales for the downstream tasks. Our approach
outperforms state-of-the-art methods on three real-world datasets, including
the latest clinical MIMIC IV dataset.