Similarity-Guided Diffusion for Long-Gap Music Inpainting
By: Sean Turland, Eloi Moliner, Vesa Välimäki
Potential Business Impact:
Fixes long missing parts in music recordings.
Music inpainting aims to reconstruct missing segments of a corrupted recording. While diffusion-based generative models improve reconstruction for medium-length gaps, they often struggle to preserve musical plausibility over multi-second gaps. We introduce Similarity-Guided Diffusion Posterior Sampling (SimDPS), a hybrid method that combines diffusion-based inference with similarity search. Candidate segments are first retrieved from a corpus based on contextual similarity, then incorporated into a modified likelihood that guides the diffusion process toward contextually consistent reconstructions. Subjective evaluation on piano music inpainting with 2-s gaps shows that the proposed SimDPS method enhances perceptual plausibility compared to unguided diffusion and frequently outperforms similarity search alone when moderately similar candidates are available. These results demonstrate the potential of a hybrid similarity approach for diffusion-based audio enhancement with long gaps.
Similar Papers
Token-based Audio Inpainting via Discrete Diffusion
Sound
Fixes broken music by filling in missing sounds.
Diffusion Timbre Transfer Via Mutual Information Guided Inpainting
Sound
Changes music's sound without retraining.
Transient Noise Removal via Diffusion-based Speech Inpainting
Audio and Speech Processing
Fixes broken or missing speech in recordings.