GenSelect: A Generative Approach to Best-of-N
By: Shubham Toshniwal , Ivan Sorokin , Aleksander Ficek and more
Potential Business Impact:
Helps AI pick the best answer by thinking.
Generative reward models with parallel sampling have enabled effective test-time scaling for reasoning tasks. Current approaches employ pointwise scoring of individual solutions or pairwise comparisons. However, pointwise methods underutilize LLMs' comparative abilities, while pairwise methods scale inefficiently with larger sampling budgets. We introduce GenSelect, where the LLM uses long reasoning to select the best solution among N candidates. This leverages LLMs' comparative strengths while scaling efficiently across parallel sampling budgets. For math reasoning, we demonstrate that reasoning models, such as QwQ and DeepSeek-R1-0528, excel at GenSelect, outperforming existing scoring approaches with simple prompting.
Similar Papers
Learning to Reason Across Parallel Samples for LLM Reasoning
Computation and Language
Makes AI smarter by checking many answers.
Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs
Machine Learning (CS)
AI learns to fix its own math mistakes.
Scaling Generative Verifiers For Natural Language Mathematical Proof Verification And Selection
Artificial Intelligence
Helps computers check math proofs for mistakes.