Prestige over merit: An adapted audit of LLM bias in peer review
By: Anthony Howell , Jieshu Wang , Luyu Du and more
Potential Business Impact:
AI favors papers from famous schools.
Large language models (LLMs) are playing an increasingly integral, though largely informal, role in scholarly peer review. Yet it remains unclear whether LLMs reproduce the biases observed in human decision-making. We adapt a resume-style audit to scientific publishing, developing a multi-role LLM simulation (editor/reviewer) that evaluates a representative set of high-quality manuscripts across the physical, biological, and social sciences under randomized author identities (institutional prestige, gender, race). The audit reveals a strong and consistent institutional-prestige bias: identical papers attributed to low-prestige affiliations face a significantly higher risk of rejection, despite only modest differences in LLM-assessed quality. To probe mechanisms, we generate synthetic CVs for the same author profiles; these encode large prestige-linked disparities and an inverted prestige-tenure gradient relative to national benchmarks. The results suggest that both domain norms and prestige-linked priors embedded in training data shape paper-level outcomes once identity is visible, converting affiliation into a decisive status cue.
Similar Papers
Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviews
Computers and Society
Finds AI reviews unfairly favor famous schools.
Justice in Judgment: Unveiling (Hidden) Bias in LLM-assisted Peer Reviews
Computers and Society
AI reviews unfairly favor famous schools and some genders.
LLM-REVal: Can We Trust LLM Reviewers Yet?
Computation and Language
AI reviewers unfairly favor AI-written papers.