A Case for a "Refutations and Critiques'' Track in Statistics Journals
By: Zhen Li
Potential Business Impact:
Fixes bad science papers with new review system.
The statistics community, which has traditionally lacked a transparent and open peer-review system, faces a challenge of inconsistent paper quality, with some published work containing substantial errors. This problem resonates with concerns raised by Schaeffer et al. (2025) regarding the rapid growth of machine learning research. They argue that peer review has proven insufficient to prevent the publication of ``misleading, incorrect, flawed or perhaps even fraudulent studies'' and that a ``dynamic self-correcting research ecosystem'' is needed. This note provides a concrete illustration of this problem by examining two published papers, Wang, Zhou and Lin (2025) and Liu et al. (2023), and exposing striking and critical errors in their proofs. The presence of such errors in major journals raises a fundamental question about the importance and verification of mathematical proofs in our field. Echoing the proposal from Schaeffer et al. (2025), we argue that reforming the peer-review system itself is likely impractical. Instead, we propose a more viable path forward: the creation of a high-profile, reputable platform, such as a ``Refutations and Critiques'' track on arXiv, to provide visibility to vital research that critically challenges prior work. Such a mechanism would be crucial for enhancing the reliability and credibility of statistical research.
Similar Papers
Insights from the ICLR Peer Review and Rebuttal Process
Computers and Society
Helps scientists get better feedback on their work.
What Drives Paper Acceptance? A Process-Centric Analysis of Modern Peer Review
Computers and Society
Makes papers more likely to be accepted.
Comment on "Deep Regression Learning with Optimal Loss Function"
Methodology
Makes science papers better by showing all feedback.