Temporal Embeddings: Scalable Self-Supervised Temporal Representation
  Learning from Spatiotemporal Data for Multimodal Computer Vision
Yi Cao Swetava Ganguli Vipul Pandey
There exists a correlation between geospatial activity temporal patterns and
type of land use. A novel self-supervised approach is proposed to stratify
landscape based on mobility activity time series. First, the time series signal
is transformed to the frequency domain and then compressed into task-agnostic
temporal embeddings by a contractive autoencoder, which preserves cyclic
temporal patterns observed in time series. The pixel-wise embeddings are
converted to image-like channels that can be used for task-based, multimodal
modeling of downstream geospatial tasks using deep semantic segmentation.
Experiments show that temporal embeddings are semantically meaningful
representations of time series data and are effective across different tasks
such as classifying residential area and commercial areas. Temporal embeddings
transform sequential, spatiotemporal motion trajectory data into semantically
meaningful image-like tensor representations that can be combined (multimodal
fusion) with other data modalities that are or can be transformed into
image-like tensor representations (for e.g., RBG imagery, graph embeddings of
road networks, passively collected imagery like SAR, etc.) to facilitate
multimodal learning in geospatial computer vision. Multimodal computer vision
is critical for training machine learning models for geospatial feature
detection to keep a geospatial mapping service up-to-date in real-time and can
significantly improve user experience and above all, user safety.