On Flow Matching KL Divergence
By: Maojiang Su , Jerry Yao-Chieh Hu , Sophia Pi and more
Potential Business Impact:
Makes AI learn data more accurately and faster.
We derive a deterministic, non-asymptotic upper bound on the Kullback-Leibler (KL) divergence of the flow-matching distribution approximation. In particular, if the $L_2$ flow-matching loss is bounded by $\epsilon^2 > 0$, then the KL divergence between the true data distribution and the estimated distribution is bounded by $A_1 \epsilon + A_2 \epsilon^2$. Here, the constants $A_1$ and $A_2$ depend only on the regularities of the data and velocity fields. Consequently, this bound implies statistical convergence rates of Flow Matching Transformers under the Total Variation (TV) distance. We show that, flow matching achieves nearly minimax-optimal efficiency in estimating smooth distributions. Our results make the statistical efficiency of flow matching comparable to that of diffusion models under the TV distance. Numerical studies on synthetic and learned velocities corroborate our theory.
Similar Papers
Distribution estimation via Flow Matching with Lipschitz guarantees
Machine Learning (Stat)
Makes AI learn faster and better.
On the minimax optimality of Flow Matching through the connection to kernel density estimation
Machine Learning (Stat)
Makes AI create realistic images faster and better.
Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching
Machine Learning (CS)
Makes AI create better pictures with less data.