Facilitating Visual Media Exploration for Blind and Low Vision Users through AI-Powered Interactive Storytelling
By: Shuchang Xu
Potential Business Impact:
Helps blind people experience stories with pictures.
Empowering blind and low vision (BLV) users to explore visual media improves content comprehension, strengthens user agency, and fulfills diverse information needs. However, most existing tools separate exploration from the main narration, which disrupts the narrative flow, increases cognitive load, and limits deep engagement with visual media. To address these challenges, my PhD research introduces the paradigm of AI-powered interactive storytelling, which leverages AI to generate interactive narratives, enabling BLV users to explore visual media within a coherent storytelling experience. I have operationalized this paradigm through three techniques: (1) Hierarchical Narrative, which supports photo-collection exploration at different levels of detail; (2) Parallel Narrative, which provides seamless access to time-synced video comments; and (3) Branching Narrative, which enables immersive navigation of 360{\deg} videos. Together, these techniques demonstrate that AI-powered interactive storytelling can effectively balance user agency with narrative coherence across diverse media formats. My future work will advance this paradigm by enabling more personalized and expressive storytelling experiences for BLV audiences.
Similar Papers
Facilitating Visual Media Exploration for Blind and Low Vision Users through AI-Powered Interactive Storytelling
Human-Computer Interaction
Helps blind people experience stories with pictures.
Creation, Critique, and Consumption: Exploring Generative AI Descriptions for Supporting Blind and Low Vision Professionals with Visual Tasks
Human-Computer Interaction
Helps blind people do jobs with pictures.
I Prompt, it Generates, we Negotiate. Exploring Text-Image Intertextuality in Human-AI Co-Creation of Visual Narratives with VLMs
Human-Computer Interaction
Helps people tell stories with AI pictures.