Beamformed 360° Sound Maps: U-Net-Driven Acoustic Source Segmentation and Localization
By: Belman Jahir Rodriguez , Sergio F. Chevtchenko , Marcelo Herrera Martinez and more
Potential Business Impact:
Pinpoints sounds from all directions accurately
We introduce a U-net model for 360{\deg} acoustic source localization formulated as a spherical semantic segmentation task. Rather than regressing discrete direction-of-arrival (DoA) angles, our model segments beamformed audio maps (azimuth and elevation) into regions of active sound presence. Using delay-and-sum (DAS) beamforming on a custom 24-microphone array, we generate signals aligned with drone GPS telemetry to create binary supervision masks. A modified U-Net, trained on frequency-domain representations of these maps, learns to identify spatially distributed source regions while addressing class imbalance via the Tversky loss. Because the network operates on beamformed energy maps, the approach is inherently array-independent and can adapt to different microphone configurations without retraining from scratch. The segmentation outputs are post-processed by computing centroids over activated regions, enabling robust DoA estimates. Our dataset includes real-world open-field recordings of a DJI Air 3 drone, synchronized with 360{\deg} video and flight logs across multiple dates and locations. Experimental results show that U-net generalizes across environments, providing improved angular precision, offering a new paradigm for dense spatial audio understanding beyond traditional Sound Source Localization (SSL).
Similar Papers
Real-Time Object Tracking with On-Device Deep Learning for Adaptive Beamforming in Dynamic Acoustic Environments
Sound
Listens to and follows specific sounds in noisy places.
AuralNet: Hierarchical Attention-based 3D Binaural Localization of Overlapping Speakers
Audio and Speech Processing
Finds sounds in 3D, even when mixed.
Sound Source Localization for Spatial Mapping of Surgical Actions in Dynamic Scenes
Sound
Helps robots "hear" where tools touch inside bodies.