AR Secretary Agent: Real-time Memory Augmentation via LLM-powered Augmented Reality Glasses
By: Raphaël A. El Haddad , Zeyu Wang , Yeonsu Shin and more
Potential Business Impact:
Helps you remember people and talks easily.
Interacting with a significant number of individuals on a daily basis is commonplace for many professionals, which can lead to challenges in recalling specific details: Who is this person? What did we talk about last time? The advant of augmented reality (AR) glasses, equipped with visual and auditory data capture capabilities, presents a solution. In our work, we implemented an AR Secretary Agent with advanced Large Language Models (LLMs) and Computer Vision technologies. This system could discreetly provide real-time information to the wearer, identifying who they are conversing with and summarizing previous discussions. To verify AR Secretary, we conducted a user study with 13 participants and showed that our technique can efficiently help users to memorize events by up to 20\% memory enhancement on our study.
Similar Papers
Designing Memory-Augmented AR Agents for Spatiotemporal Reasoning in Personalized Task Assistance
Artificial Intelligence
Helps smart glasses remember your past to help you.
ChatAR: Conversation Support using Large Language Model and Augmented Reality
Human-Computer Interaction
Helps you talk better by showing hidden info.
Teaching LLMs to See and Guide: Context-Aware Real-Time Assistance in Augmented Reality
Human-Computer Interaction
Helps AR/VR assistants understand what you're doing.