Building a Foundation Model for Trajectory from Scratch
By: Gaspard Merten, Mahmoud Sakr, Gilles Dejaegere
Potential Business Impact:
Teaches computers to predict where things will go.
Foundation models are transformative in artificial intelligence, but building them from scratch, especially for mobility trajectories, is not yet clear or documented. This tutorial bridges this gap by demonstrating the steps and code of a minimal implementation of a trajectory-focused foundation model starting from GPT-2. Through a concise, step-by-step, code-driven process, we demonstrate adapting GPT-2 for spatiotemporal data. We then review and compare representative trajectory foundation models, such as TrajFM and TrajGPT, highlighting their architectural innovations and differences. Additionally, we introduce complementary techniques from related domains, like TimesFM's patching approach. Targeted at researchers and practitioners, this tutorial aims to explain the concepts and terminology of foundation models, at the implementation level. We find it timely and indispensable to create this educational material in order to support the SIGSPATIAL community in building and evaluating mobility foundation models, enhancing both research clarity and peer-review effectiveness in mobility AI.
Similar Papers
From Prediction to Understanding: Will AI Foundation Models Transform Brain Science?
Neurons and Cognition
Helps AI understand how brains work.
AI Foundation Model for Time Series with Innovations Representation
Machine Learning (Stat)
Helps machines predict future events accurately.
Timeseries Foundation Models for Mobility: A Benchmark Comparison with Traditional and Deep Learning Models
Machine Learning (CS)
Predicts city bike use hours ahead.