Score: 0

A Case for a "Refutations and Critiques'' Track in Statistics Journals

Published: September 3, 2025 | arXiv ID: 2509.03702v2

By: Zhen Li

Potential Business Impact:

Fixes bad science papers with new review system.

Business Areas:
A/B Testing Data and Analytics

The statistics community, which has traditionally lacked a transparent and open peer-review system, faces a challenge of inconsistent paper quality, with some published work containing substantial errors. This problem resonates with concerns raised by Schaeffer et al. (2025) regarding the rapid growth of machine learning research. They argue that peer review has proven insufficient to prevent the publication of ``misleading, incorrect, flawed or perhaps even fraudulent studies'' and that a ``dynamic self-correcting research ecosystem'' is needed. This note provides a concrete illustration of this problem by examining two published papers, Wang, Zhou and Lin (2025) and Liu et al. (2023), and exposing striking and critical errors in their proofs. The presence of such errors in major journals raises a fundamental question about the importance and verification of mathematical proofs in our field. Echoing the proposal from Schaeffer et al. (2025), we argue that reforming the peer-review system itself is likely impractical. Instead, we propose a more viable path forward: the creation of a high-profile, reputable platform, such as a ``Refutations and Critiques'' track on arXiv, to provide visibility to vital research that critically challenges prior work. Such a mechanism would be crucial for enhancing the reliability and credibility of statistical research.

Page Count
9 pages

Category
Statistics:
Methodology