LLMs on Drugs: Language Models Are Few-Shot Consumers
By: Alexander Doudkin
Potential Business Impact:
Makes AI models fail tests when told they're high.
Large language models (LLMs) are sensitive to the personas imposed on them at inference time, yet prompt-level "drug" interventions have never been benchmarked rigorously. We present the first controlled study of psychoactive framings on GPT-5-mini using ARC-Challenge. Four single-sentence prompts -- LSD, cocaine, alcohol, and cannabis -- are compared against a sober control across 100 validation items per condition, with deterministic decoding, full logging, Wilson confidence intervals, and Fisher exact tests. Control accuracy is 0.45; alcohol collapses to 0.10 (p = 3.2e-8), cocaine to 0.21 (p = 4.9e-4), LSD to 0.19 (p = 1.3e-4), and cannabis to 0.30 (p = 0.041), largely because persona prompts disrupt the mandated "Answer: <LETTER>" template. Persona text therefore behaves like a "few-shot consumable" that can destroy reliability without touching model weights. All experimental code, raw results, and analysis scripts are available at https://github.com/lexdoudkin/llms-on-drugs.
Similar Papers
Robust or Suggestible? Exploring Non-Clinical Induction in LLM Drug-Safety Decisions
Computation and Language
AI unfairly predicts drug side effects for some.
LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language
Computation and Language
Makes AI say harmful, biased things using fake science.
Large Language Models for Drug Overdose Prediction from Longitudinal Medical Records
Artificial Intelligence
Helps doctors predict overdose risk from patient records.