EASL: Multi-Emotion Guided Semantic Disentanglement for Expressive Sign Language Generation
By: Yanchao Zhao , Jihao Zhu , Yu Liu and more
Potential Business Impact:
Makes sign language videos show feelings.
Large language models have revolutionized sign language generation by automatically transforming text into high-quality sign language videos, providing accessible communication for the Deaf community. However, existing LLM-based approaches prioritize semantic accuracy while overlooking emotional expressions, resulting in outputs that lack naturalness and expressiveness. We propose EASL (Emotion-Aware Sign Language), a multi-emotion-guided generation architecture for fine-grained emotional integration. We introduce emotion-semantic disentanglement modules with progressive training to separately extract semantic and affective features. During pose decoding, the emotional representations guide semantic interaction to generate sign poses with 7-class emotion confidence scores, enabling emotional expression recognition. Experimental results demonstrate that EASL achieves pose accuracy superior to all compared baselines by integrating multi-emotion information and effectively adapts to diffusion models to generate expressive sign language videos.
Similar Papers
Emotion-Enhanced Multi-Task Learning with LLMs for Aspect Category Sentiment Analysis
Computation and Language
Teaches computers to understand feelings behind words.
Seeing is Believing: Emotion-Aware Audio-Visual Language Modeling for Expressive Speech Generation
Computation and Language
Makes computer voices sound more real.
Seeing is Believing: Emotion-Aware Audio-Visual Language Modeling for Expressive Speech Generation
Computation and Language
Makes computers talk with real-life facial expressions.