Conformal Correction for Efficiency May be at Odds with Entropy
By: Senrong Xu , Tianyu Wang , Zenan Li and more
Potential Business Impact:
Makes AI predictions more trustworthy and faster.
Conformal prediction (CP) provides a comprehensive framework to produce statistically rigorous uncertainty sets for black-box machine learning models. To further improve the efficiency of CP, conformal correction is proposed to fine-tune or wrap the base model with an extra module using a conformal-aware inefficiency loss. In this work, we empirically and theoretically identify a trade-off between the CP efficiency and the entropy of model prediction. We then propose an entropy-constrained conformal correction method, exploring a better Pareto optimum between efficiency and entropy. Extensive experimental results on both computer vision and graph datasets demonstrate the efficacy of the proposed method. For instance, it can significantly improve the efficiency of state-of-the-art CP methods by up to 34.4%, given an entropy threshold.
Similar Papers
Cost-Sensitive Conformal Training with Provably Controllable Learning Bounds
Machine Learning (CS)
Makes AI predictions more accurate and reliable.
Efficient Conformal Prediction for Regression Models under Label Noise
Machine Learning (CS)
Makes medical scans more trustworthy with bad data.
Reliable Statistical Guarantees for Conformal Predictors with Small Datasets
Machine Learning (CS)
Makes AI predictions more trustworthy, even with little data.