Iterative Tilting for Diffusion Fine-Tuning
By: Jean Pachebat , Giovanni Conforti , Alain Durmus and more
Potential Business Impact:
Makes AI create better art by learning from mistakes.
We introduce iterative tilting, a gradient-free method for fine-tuning diffusion models toward reward-tilted distributions. The method decomposes a large reward tilt $\exp(λr)$ into $N$ sequential smaller tilts, each admitting a tractable score update via first-order Taylor expansion. This requires only forward evaluations of the reward function and avoids backpropagating through sampling chains. We validate on a two-dimensional Gaussian mixture with linear reward, where the exact tilted distribution is available in closed form.
Similar Papers
Tilt Matching for Scalable Sampling and Fine-Tuning
Machine Learning (Stat)
Makes AI create better pictures and art.
Test-time scaling of diffusions with flow maps
Machine Learning (CS)
Makes AI create better pictures by guiding its choices.
Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Design
Machine Learning (CS)
Designs new proteins and medicines faster.