Regions are Who Walk Them: a Large Pre-trained Spatiotemporal Model
  Based on Human Mobility for Ubiquitous Urban Sensing
Ruixing Zhang Liangzhe Han Leilei Sun Yunqi Liu Jibin Wang Weifeng Lv
User profiling and region analysis are two tasks of significant commercial
value. However, in practical applications, modeling different features
typically involves four main steps: data preparation, data processing, model
establishment, evaluation, and optimization. This process is time-consuming and
labor-intensive. Repeating this workflow for each feature results in abundant
development time for tasks and a reduced overall volume of task development.
Indeed, human mobility data contains a wealth of information. Several
successful cases suggest that conducting in-depth analysis of population
movement data could potentially yield meaningful profiles about users and
areas. Nonetheless, most related works have not thoroughly utilized the
semantic information within human mobility data and trained on a fixed number
of the regions. To tap into the rich information within population movement,
based on the perspective that Regions Are Who walk them, we propose a large
spatiotemporal model based on trajectories (RAW). It possesses the following
characteristics: 1) Tailored for trajectory data, introducing a GPT-like
structure with a parameter count of up to 1B; 2) Introducing a spatiotemporal
fine-tuning module, interpreting trajectories as collection of users to derive
arbitrary region embedding. This framework allows rapid task development based
on the large spatiotemporal model. We conducted extensive experiments to
validate the effectiveness of our proposed large spatiotemporal model. It's
evident that our proposed method, relying solely on human mobility data without
additional features, exhibits a certain level of relevance in user profiling
and region analysis. Moreover, our model showcases promising predictive
capabilities in trajectory generation tasks based on the current state,
offering the potential for further innovative work utilizing this large
spatiotemporal model.