In this paper, we present a new deep learning architecture for addressing the problem of supervised learning with sparse and irregularly sampled multivariate time series. The architecture is based on the use of a semi-parametric interpolation network followed by the application of a prediction network. The interpolation network allows for information to be shared across multiple dimensions of a multivariate time series during the interpolation stage, while any standard deep learning model can be used for the prediction network. This work is motivated by the analysis of physiological time series data in electronic health records, which are sparse, irregularly sampled, and multivariate. We investigate the performance of this architecture on both classification and regression tasks, showing that our approach outperforms a range of baseline and recently proposed models.
 Over the last several years, there has been significant progress in developing specialized models and architectures that can accommodate sparse and irregularly sampled time series as input BID25 BID20 BID21 BID12 BID4 ). An irregularly sampled time series is a sequence of samples with irregular intervals between their observation times. Irregularly sampled data are considered to be sparse when the intervals between successive observations are often large. Of particular interest in the supervised learning setting are methods that perform end-to-end learning directly using multivariate sparse and irregularly sampled time series as input without the need for a separate interpolation or imputation step.In this work, we present a new model architecture for supervised learning with multivariate sparse and irregularly sampled data: Interpolation-Prediction Networks. The architecture is based on the use of several semi-parametric interpolation layers organized into an interpolation network, followed by the application of a prediction network that can leverage any standard deep learning model. In this work, we use GRU networks BID6 as the prediction network.The interpolation network allows for information contained in each input time series to contribute to the interpolation of all other time series in the model. The parameters of the interpolation and prediction networks are learned end-to-end via a composite objective function consisting of supervised and unsupervised components. The interpolation network serves the same purpose as the multivariate Gaussian process used in the work of BID12 , but remove the restrictions associated with the need for a positive definite covariance matrix.Our approach also allows us to compute an explicit multi-timescale representation of the input time series, which we use to isolate information about transients (short duration events) from broader trends. Similar to the work of BID21 and BID4 , our architecture also explicitly leverages a separate information channel related to patterns of observation times. However, our representation uses a semi-parametric intensity function representation of this information that is more closely related to the work of Lasko (2014) on modeling medical event point processes.Our architecture thus produces three output time series for each input time series: a smooth interpolation modeling broad trends in the input, a short time-scale interpolation modeling transients, and an intensity function modeling local observation frequencies.This work is motivated by problems in the analysis of electronic health records (EHRs) BID25 BID21 BID12 BID4 ). It remains rare for hospital systems to capture dense physiological data streams. Instead, it is common for the physiological time series data in electronic health records to be both sparse and irregularly sampled. The additional issue of the lack of alignment in the observation times across physiological variables is also very common.We evaluate the proposed architecture on two datasets for both classification and regression tasks. Our approach outperforms a variety of simple baseline models as well as the basic and advanced GRU models introduced by BID4 across several metrics. We also compare our model with to the Gaussian process adapter BID19 and multi-task Gaussian process RNN classifier BID12 . Further, we perform full ablation testing of the information channels our architecture can produce to assess their impact on classification and regression performance. In this paper, we have presented a new framework for dealing with the problem of supervised learning in the presence of sparse and irregularly sampled time series. The proposed framework is fully modular. It uses an interpolation network to accommodate the complexity that results from using sparse and irregularly sampled data as supervised learning inputs, followed by the application of a prediction network that operates over the regularly spaced and fully observed, multi-channel output provided by the interpolation network. The proposed approach also addresses some difficulties with prior approaches including the complexity of the Gaussian process interpolation layers used in BID19 BID12 , and the lack of modularity in the approach of BID4 . Our framework also introduces novel elements including the use of semi-parametric, feed-forward interpolation layers, and the decomposition of an irregularly sampled input time series into multi-ple distinct information channels. Our results show statistically significant improvements for both classification and regression tasks over a range of baseline and state-of-the-art methods.Ruslan This data set contains sparse and irregularly sampled physiological signals, medications, diagnostic codes, inhospital mortality, length of stay and more. We focus on predicting in-hospital mortality and length of stay using the first 48 hours of data. We extracted 12 standard physiological variables from each of the 53,211 records obtained after removing hospital admission records with length of stay less than 48 hours. TAB2 shows the features, sampling rates (per hour) and their missingness information computed using the union of all time stamps that exist in any dimension of the input time series. In our experiments, each admission record corresponds to one data case (s n , y n ). Each data case n consists of a sparse and irregularly sampled time series s n with D = 12 dimensions. Each dimension d of s n corresponds to one of the 12 vital sign time series mentioned above. In the case of classification, y n is a binary indicator where y n = 1 indicates that the patient died at any point within the hospital stay following the first 48 hours and y n = 0 indicates that the patient was discharged at any point after the first 48 hours. There are 4310 (8.1%) patients with a y n = 1 mortality label. The complete data set is D = {(s n , y n )|n = 1, ..., N }, and there are N = 53, 211 data cases. The goal in the classification task is to learn a classification function g of the form y n ← g(s n ) whereŷ n is a discrete value.In the case of regression, y n is a real-valued regression target corresponding to the length of stay. Since the data set includes some very long stay durations, we let y n represent the log of the length of stay in days for all models. We convert back from the log number of days to the number of days when reporting results. The complete data set is again D = {(s n , y n )|n = 1, ..., N } with N = 53, 211 data cases (we again require 48 hours worth of data). The goal in the regression task is to learn a regression function g of the formŷ n ← g(s n ) whereŷ n is a continuous value.
This paper presents a new deep learning architecture for addressing the problem of supervised learning with sparse and irregularly sampled multivariate time series. Proposes a framework for making predictions on sparse, irregularly sampled time-series data using an interpolation module that models the missing values in using smooth interpolation, non-smooth interpolation, and intensity.  Solves the problem of supervised learning with sparse and irregularly sampled multivariate time series using a semi-parametric interpolation network followed by a prediction network.