STER-VLM: Spatio-Temporal With Enhanced Reference Vision-Language Models
By: Tinh-Anh Nguyen-Nhu , Triet Dao Hoang Minh , Dat To-Thanh and more
Potential Business Impact:
Helps self-driving cars understand traffic better.
Vision-language models (VLMs) have emerged as powerful tools for enabling automated traffic analysis; however, current approaches often demand substantial computational resources and struggle with fine-grained spatio-temporal understanding. This paper introduces STER-VLM, a computationally efficient framework that enhances VLM performance through (1) caption decomposition to tackle spatial and temporal information separately, (2) temporal frame selection with best-view filtering for sufficient temporal information, and (3) reference-driven understanding for capturing fine-grained motion and dynamic context and (4) curated visual/textual prompt techniques. Experimental results on the WTS \cite{kong2024wts} and BDD \cite{BDD} datasets demonstrate substantial gains in semantic richness and traffic scene interpretation. Our framework is validated through a decent test score of 55.655 in the AI City Challenge 2025 Track 2, showing its effectiveness in advancing resource-efficient and accurate traffic analysis for real-world applications.
Similar Papers
VLM4D: Towards Spatiotemporal Awareness in Vision Language Models
CV and Pattern Recognition
Tests AI's grasp of video movements and fixes gaps
VLM4D: Towards Spatiotemporal Awareness in Vision Language Models
CV and Pattern Recognition
Helps computers understand moving objects like humans do.
Vision-LLMs for Spatiotemporal Traffic Forecasting
Machine Learning (CS)
Predicts city traffic jams before they happen.