Uncertainty Under the Curve: A Sequence-Level Entropy Area Metric for Reasoning LLM
By: Yongfu Zhu , Lin Sun , Guangxiang Zhao and more
Potential Business Impact:
Helps AI learn better by spotting confusing answers.
In this work, we introduce Entropy Area Score (EAS), a simple yet effective metric to quantify uncertainty in the answer generation process of reasoning large language models (LLMs). EAS requires neither external models nor repeated sampling, it integrates token-level predictive entropy from the model itself to capture the evolution of uncertainty during generation. Empirical results show that EAS is strongly correlated with answer entropy across models and datasets. In training data selection, EAS identifies high-potential samples and consistently outperforms Pass Rate filtering under equal sample budgets, improving student model accuracy on math benchmarks. EAS is both efficient and interpretable, offering a practical tool for uncertainty modeling and data quality assessment in LLM training.
Similar Papers
Entropy-Aware Speculative Decoding Toward Improved LLM Reasoning
Computation and Language
Makes AI smarter than its own training.
When an LLM is apprehensive about its answers -- and when its uncertainty is justified
Computation and Language
Helps AI know when it's guessing wrong.
Beyond the Score: Uncertainty-Calibrated LLMs for Automated Essay Assessment
Computation and Language
Helps computers grade essays with confidence.