Low-rank Adaptation for Spatio-Temporal Forecasting
Weilin Ruan Wei Chen Xilin Dang Jianxiang Zhou Weichuang Li Xu Liu Yuxuan Liang
Spatio-temporal forecasting is crucial in real-world dynamic systems,
predicting future changes using historical data from diverse locations.
Existing methods often prioritize the development of intricate neural networks
to capture the complex dependencies of the data, yet their accuracy fails to
show sustained improvement. Besides, these methods also overlook node
heterogeneity, hindering customized prediction modules from handling diverse
regional nodes effectively. In this paper, our goal is not to propose a new
model but to present a novel low-rank adaptation framework as an off-the-shelf
plugin for existing spatial-temporal prediction models, termed ST-LoRA, which
alleviates the aforementioned problems through node-level adjustments.
Specifically, we first tailor a node adaptive low-rank layer comprising
multiple trainable low-rank matrices. Additionally, we devise a multi-layer
residual fusion stacking module, injecting the low-rank adapters into predictor
modules of various models. Across six real-world traffic datasets and six
different types of spatio-temporal prediction models, our approach minimally
increases the parameters and training time of the original models by less than
4%, still achieving consistent and sustained performance enhancement.