Calibrating Generative Models
By: Henry D. Smith, Nathaniel L. Diamant, Brian L. Trippe
Potential Business Impact:
Makes AI more honest about what it knows.
Generative models frequently suffer miscalibration, wherein class probabilities and other statistics of the sampling distribution deviate from desired values. We frame calibration as a constrained optimization problem and seek the closest model in Kullback-Leibler divergence satisfying calibration constraints. To address the intractability of imposing these constraints exactly, we introduce two surrogate objectives for fine-tuning: (1) the relax loss, which replaces the constraint with a miscalibration penalty, and (2) the reward loss, which converts calibration into a reward fine-tuning problem. We demonstrate that these approaches substantially reduce calibration error across hundreds of simultaneous constraints and models with up to one billion parameters, spanning applications in protein design, image generation, and language modeling.
Similar Papers
Efficient Calibration for Decision Making
Machine Learning (CS)
Makes AI predictions more trustworthy and useful.
Calibrating Geophysical Predictions under Constrained Probabilistic Distributions
Atmospheric and Oceanic Physics
Improves weather forecasts by learning from past patterns.
On the Entropy Calibration of Language Models
Computation and Language
Fixes AI writing so it doesn't get worse.