Do language models accommodate their users? A study of linguistic convergence
By: Terra Blevins, Susanne Schmalwieser, Benjamin Roth
Potential Business Impact:
Computers copy how you talk in chats.
While large language models (LLMs) are generally considered proficient in generating language, how similar their language usage is to that of humans remains understudied. In this paper, we test whether models exhibit linguistic convergence, a core pragmatic element of human language communication, asking: do models adapt, or converge, to the linguistic patterns of their user? To answer this, we systematically compare model completions of exisiting dialogues to the original human responses across sixteen language models, three dialogue corpora, and a variety of stylometric features. We find that models strongly converge to the conversation's style, often significantly overfitting relative to the human baseline. While convergence patterns are often feature-specific, we observe consistent shifts in convergence across modeling settings, with instruction-tuned and larger models converging less than their pretrained counterparts. Given the differences between human and model convergence patterns, we hypothesize that the underlying mechanisms for these behaviors are very different.
Similar Papers
LLMs syntactically adapt their language use to their conversational partner
Computation and Language
Computers copy how people talk to each other.
Mind the Gap: Linguistic Divergence and Adaptation Strategies in Human-LLM Assistant vs. Human-Human Interactions
Computation and Language
Teaches chatbots to talk like people do.
How human is the machine? Evidence from 66,000 Conversations with Large Language Models
Human-Computer Interaction
AI sometimes thinks differently than people.