Visionary Co-Driver: Enhancing Driver Perception of Potential Risks with LLM and HUD
By: Wei Xiang , Ziyue Lei , Jie Wang and more
Potential Business Impact:
Helps drivers see hidden dangers on the road.
Drivers' perception of risky situations has always been a challenge in driving. Existing risk-detection methods excel at identifying collisions but face challenges in assessing the behavior of road users in non-collision situations. This paper introduces Visionary Co-Driver, a system that leverages large language models to identify non-collision roadside risks and alert drivers based on their eye movements. Specifically, the system combines video processing algorithms and LLMs to identify potentially risky road users. These risks are dynamically indicated on an adaptive heads-up display interface to enhance drivers' attention. A user study with 41 drivers confirms that Visionary Co-Driver improves drivers' risk perception and supports their recognition of roadside risks.
Similar Papers
Driver Assistant: Persuading Drivers to Adjust Secondary Tasks Using Large Language Models
Human-Computer Interaction
Helps drivers stay focused while cars drive themselves.
Toward Automatic Safe Driving Instruction: A Large-Scale Vision Language Model Approach
CV and Pattern Recognition
Helps cars watch drivers and roads for safety.
Bridging Human Oversight and Black-box Driver Assistance: Vision-Language Models for Predictive Alerting in Lane Keeping Assist Systems
Robotics
Warns drivers before car steering fails.