On The Hidden Biases of Flow Matching Samplers
By: Soon Hoe Lim
We study the implicit bias of flow matching (FM) samplers via the lens of empirical flow matching. Although population FM may produce gradient-field velocities resembling optimal transport (OT), we show that the empirical FM minimizer is almost never a gradient field, even when each conditional flow is. Consequently, empirical FM is intrinsically energetically suboptimal. In view of this, we analyze the kinetic energy of generated samples. With Gaussian sources, both instantaneous and integrated kinetic energies exhibit exponential concentration, while heavy-tailed sources lead to polynomial tails. These behaviors are governed primarily by the choice of source distribution rather than the data. Overall, these notes provide a concise mathematical account of the structural and energetic biases arising in empirical FM.
Similar Papers
Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching
Machine Learning (CS)
Makes AI create better pictures with less data.
On the continuity of flows
Machine Learning (CS)
Makes AI create new things with tricky shapes.
On the minimax optimality of Flow Matching through the connection to kernel density estimation
Machine Learning (Stat)
Makes AI create realistic images faster and better.