GATGPT: A Pre-trained Large Language Model with Graph Attention Network
  for Spatiotemporal Imputation
Yakun Chen Xianzhi Wang Guandong Xu
The analysis of spatiotemporal data is increasingly utilized across diverse
domains, including transportation, healthcare, and meteorology. In real-world
settings, such data often contain missing elements due to issues like sensor
malfunctions and data transmission errors. The objective of spatiotemporal
imputation is to estimate these missing values by understanding the inherent
spatial and temporal relationships in the observed multivariate time series.
Traditionally, spatiotemporal imputation has relied on specific, intricate
architectures designed for this purpose, which suffer from limited
applicability and high computational complexity. In contrast, our approach
integrates pre-trained large language models (LLMs) into spatiotemporal
imputation, introducing a groundbreaking framework, GATGPT. This framework
merges a graph attention mechanism with LLMs. We maintain most of the LLM
parameters unchanged to leverage existing knowledge for learning temporal
patterns, while fine-tuning the upper layers tailored to various applications.
The graph attention component enhances the LLM's ability to understand spatial
relationships. Through tests on three distinct real-world datasets, our
innovative approach demonstrates comparable results to established deep
learning benchmarks.