Towards A Foundation Model For Trajectory Intelligence
Alameen Najjar
We present the results of training a large trajectory model using real-world
user check-in data. Our approach follows a pre-train and fine-tune paradigm,
where a base model is pre-trained via masked trajectory modeling and then
adapted through fine-tuning for various downstream tasks. To address challenges
posed by noisy data and large spatial vocabularies, we propose a novel spatial
tokenization block. Our empirical analysis utilizes a comprehensive dataset of
over 2 billion check-ins generated by more than 6 million users. Through
fine-tuning on 3 downstream tasks we demonstrate that our base model has
effectively learned valuable underlying patterns in raw data, enabling its
application in meaningful trajectory intelligence tasks. Despite some
limitations, we believe this work represents an important step forward in the
realization of a foundation model for trajectory intelligence.