A Free Probabilistic Framework for Denoising Diffusion Models: Entropy, Transport, and Reverse Processes
By: Swagatam Das
Potential Business Impact:
Creates new data from complex patterns.
This paper develops a rigorous probabilistic framework that extends denoising diffusion models to the setting of noncommutative random variables. Building on Voiculescu's theory of free entropy and free Fisher information, we formulate diffusion and reverse processes governed by operator-valued stochastic dynamics whose spectral measures evolve by additive convolution. Using tools from free stochastic analysis -- including a Malliavin calculus and a Clark--Ocone representation -- we derive the reverse-time stochastic differential equation driven by the conjugate variable, the analogue of the classical score function. The resulting dynamics admit a gradient-flow structure in the noncommutative Wasserstein space, establishing an information-geometric link between entropy production, transport, and deconvolution. We further construct a variational scheme analogous to the Jordan--Kinderlehrer--Otto (JKO) formulation and prove convergence toward the semicircular equilibrium. The framework provides functional inequalities (free logarithmic Sobolev, Talagrand, and HWI) that quantify entropy dissipation and Wasserstein contraction. These results unify diffusion-based generative modeling with the geometry of operator-valued information, offering a mathematical foundation for generative learning on structured and high-dimensional data.
Similar Papers
A Free Probabilistic Framework for Denoising Diffusion Models: Entropy, Transport, and Reverse Processes
Probability
Makes computers create complex data from noise.
A Unified Approach to Analysis and Design of Denoising Markov Models
Machine Learning (CS)
Creates new realistic images from noise.
A Fourier Space Perspective on Diffusion Models
Machine Learning (Stat)
Makes AI create clearer, more detailed pictures.