MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
By: Hardik B. Sailor , Aw Ai Ti , Chen Fang Yih Nancy and more
Potential Business Impact:
Helps computers understand emotions in voices.
We present MERaLiON-SER, a robust speech emotion recognition model de- signed for English and Southeast Asian languages. The model is trained using a hybrid objective combining weighted categorical cross-entropy and Concordance Correlation Coefficient (CCC) losses for joint discrete and dimensional emotion modelling. This dual approach enables the model to capture both the distinct categories of emotion (like happy or angry) and the fine-grained, such as arousal (intensity), valence (positivity/negativity), and dominance (sense of control), lead- ing to a more comprehensive and robust representation of human affect. Extensive evaluations across multilingual Singaporean languages (English, Chinese, Malay, and Tamil ) and other public benchmarks show that MERaLiON-SER consistently surpasses both open-source speech encoders and large Audio-LLMs. These results underscore the importance of specialised speech-only models for accurate paralin- guistic understanding and cross-lingual generalisation. Furthermore, the proposed framework provides a foundation for integrating emotion-aware perception into future agentic audio systems, enabling more empathetic and contextually adaptive multimodal reasoning.
Similar Papers
MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
Sound
Lets computers understand emotions in voices.
MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
Sound
Lets computers understand emotions in voices.
M4SER: Multimodal, Multirepresentation, Multitask, and Multistrategy Learning for Speech Emotion Recognition
Human-Computer Interaction
Helps computers understand feelings from voices better.