Spatiotemporal Transformer for Imputing Sparse Data: A Deep Learning
  Approach
Kehui Yao Jingyi Huang Jun Zhu
Effective management of environmental resources and agricultural
sustainability heavily depends on accurate soil moisture data. However,
datasets like the SMAP/Sentinel-1 soil moisture product often contain missing
values across their spatiotemporal grid, which poses a significant challenge.
This paper introduces a novel Spatiotemporal Transformer model (ST-Transformer)
specifically designed to address the issue of missing values in sparse
spatiotemporal datasets, particularly focusing on soil moisture data. The
ST-Transformer employs multiple spatiotemporal attention layers to capture the
complex spatiotemporal correlations in the data and can integrate additional
spatiotemporal covariates during the imputation process, thereby enhancing its
accuracy. The model is trained using a self-supervised approach, enabling it to
autonomously predict missing values from observed data points. Our model's
efficacy is demonstrated through its application to the SMAP 1km soil moisture
data over a 36 x 36 km grid in Texas. It showcases superior accuracy compared
to well-known imputation methods. Additionally, our simulation studies on other
datasets highlight the model's broader applicability in various spatiotemporal
imputation tasks.