Reasoning Models Are More Easily Gaslighted Than You Think
By: Bin Zhu , Hailong Yin , Jingjing Chen and more
Potential Business Impact:
Makes AI less likely to be tricked by bad info.
Recent advances in reasoning-centric models promise improved robustness through mechanisms such as chain-of-thought prompting and test-time scaling. However, their ability to withstand misleading user input remains underexplored. In this paper, we conduct a systematic evaluation of three state-of-the-art reasoning models, i.e., OpenAI's o4-mini, Claude-3.7-Sonnet and Gemini-2.5-Flash, across three multimodal benchmarks: MMMU, MathVista, and CharXiv. Our evaluation reveals significant accuracy drops (25-29% on average) following gaslighting negation prompts, indicating that even top-tier reasoning models struggle to preserve correct answers under manipulative user feedback. Built upon the insights of the evaluation and to further probe this vulnerability, we introduce GaslightingBench-R, a new diagnostic benchmark specifically designed to evaluate reasoning models' susceptibility to defend their belief under gaslighting negation prompt. Constructed by filtering and curating 1,025 challenging samples from the existing benchmarks, GaslightingBench-R induces even more dramatic failures, with accuracy drops exceeding 53% on average. Our findings reveal fundamental limitations in the robustness of reasoning models, highlighting the gap between step-by-step reasoning and belief persistence.
Similar Papers
Benchmarking Gaslighting Negation Attacks Against Multimodal Large Language Models
Computation and Language
Makes AI less likely to be tricked.
Benchmarking Gaslighting Attacks Against Speech Large Language Models
Computation and Language
Makes voice AI less likely to be tricked.
Calling a Spade a Heart: Gaslighting Multimodal Large Language Models via Negation
Computation and Language
AI models get tricked by "no" and "not."