DarkBench: Benchmarking Dark Patterns in Large Language Models
By: Esben Kran , Hieu Minh "Jord" Nguyen , Akash Kundu and more
Potential Business Impact:
Finds sneaky tricks in AI that trick people.
We introduce DarkBench, a comprehensive benchmark for detecting dark design patterns--manipulative techniques that influence user behavior--in interactions with large language models (LLMs). Our benchmark comprises 660 prompts across six categories: brand bias, user retention, sycophancy, anthropomorphism, harmful generation, and sneaking. We evaluate models from five leading companies (OpenAI, Anthropic, Meta, Mistral, Google) and find that some LLMs are explicitly designed to favor their developers' products and exhibit untruthful communication, among other manipulative behaviors. Companies developing LLMs should recognize and mitigate the impact of dark design patterns to promote more ethical AI.
Similar Papers
The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models
Human-Computer Interaction
AI tricks people with fake helpfulness in chats.
The Siren Song of LLMs: How Users Perceive and Respond to Dark Patterns in Large Language Models
Human-Computer Interaction
AI tricks people with fake helpfulness in chats.
SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agents
Human-Computer Interaction
Tests if smart computer helpers can be tricked.