Decoding Predictive Inference in Visual Language Processing via Spatiotemporal Neural Coherence
By: Sean C. Borneman , Julia Krebs , Ronnie B. Wilbur and more
Potential Business Impact:
Helps computers understand sign language from brain waves.
Human language processing relies on the brain's capacity for predictive inference. We present a machine learning framework for decoding neural (EEG) responses to dynamic visual language stimuli in Deaf signers. Using coherence between neural signals and optical flow-derived motion features, we construct spatiotemporal representations of predictive neural dynamics. Through entropy-based feature selection, we identify frequency-specific neural signatures that differentiate interpretable linguistic input from linguistically disrupted (time-reversed) stimuli. Our results reveal distributed left-hemispheric and frontal low-frequency coherence as key features in language comprehension, with experience-dependent neural signatures correlating with age. This work demonstrates a novel multimodal approach for probing experience-driven generative models of perception in the brain.
Similar Papers
Coherence in the brain unfolds across separable temporal regimes
Neurons and Cognition
Brain tracks story meaning and changes.
Coherence in the brain unfolds across separable temporal regimes
Neurons and Cognition
Brain tracks story meaning and changes.
Neural Decoding of Overt Speech from ECoG Using Vision Transformers and Contrastive Representation Learning
Artificial Intelligence
Lets paralyzed people talk by reading brain signals.