Sparse Deep Learning for Time Series Data: Theory and Applications
Mingxuan Zhang Yan Sun Faming Liang
Sparse deep learning has become a popular technique for improving the
performance of deep neural networks in areas such as uncertainty
quantification, variable selection, and large-scale network compression.
However, most existing research has focused on problems where the observations
are independent and identically distributed (i.i.d.), and there has been little
work on the problems where the observations are dependent, such as time series
data and sequential data in natural language processing. This paper aims to
address this gap by studying the theory for sparse deep learning with dependent
data. We show that sparse recurrent neural networks (RNNs) can be consistently
estimated, and their predictions are asymptotically normally distributed under
appropriate assumptions, enabling the prediction uncertainty to be correctly
quantified. Our numerical results show that sparse deep learning outperforms
state-of-the-art methods, such as conformal predictions, in prediction
uncertainty quantification for time series data. Furthermore, our results
indicate that the proposed method can consistently identify the autoregressive
order for time series data and outperform existing methods in large-scale model
compression. Our proposed method has important practical implications in fields
such as finance, healthcare, and energy, where both accurate point estimates
and prediction uncertainty quantification are of concern.