Audio Question Answering with GRPO-Based Fine-Tuning and Calibrated Segment-Level Predictions
By: Marcel Gibier , Nolwenn Celton , Raphaël Duroselle and more
Potential Business Impact:
Answers questions about sounds using AI.
In this report, we describe our submission to Track 5 of the DCASE 2025 Challenge for the task of Audio Question Answering(AQA). Our system leverages the SSL backbone BEATs to extract frame-level audio features, which are then processed by a classification head to generate segment-level predictions of acoustic events, following the Audioset ontology. These segment-level predictions are subsequently calibrated before producing event-level predictions. Finally, these predictions are incorporated into a structured prompt, along with the question and candidate answers. This prompt is then fed to a fine-tuned version of Qwen2.5-7B-Instruct, trained using the GRPO algorithm with a simple reward function. Our method achieves an accuracy of 62.6 % on the development set, demonstrating the effectiveness of combining acoustic event reasoning with instruction-tuned large language models for AQA.
Similar Papers
Multi-Domain Audio Question Answering Toward Acoustic Content Reasoning in The DCASE 2025 Challenge
Sound
Teaches computers to answer questions about sounds.
AURA Score: A Metric For Holistic Audio Question Answering Evaluation
Audio and Speech Processing
Helps computers understand spoken answers better.
Question-Aware Gaussian Experts for Audio-Visual Question Answering
CV and Pattern Recognition
Helps computers answer questions about videos better.