Less is more: Probabilistic reduction is best explained by small-scale predictability measures
By: Cassandra L. Jacobs , Andrés Buxó-Lugo , Anna K. Taylor and more
Potential Business Impact:
Finds how much language models need to think.
The primary research questions of this paper center on defining the amount of context that is necessary and/or appropriate when investigating the relationship between language model probabilities and cognitive phenomena. We investigate whether whole utterances are necessary to observe probabilistic reduction and demonstrate that n-gram representations suffice as cognitive units of planning.
Similar Papers
Readability $\ne$ Learnability: Rethinking the Role of Simplicity in Training Small Language Models
Computation and Language
Simple words don't always make computers learn better.
Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale
Computation and Language
Makes AI understand words by how often they're used.
Exploring the Potential for Large Language Models to Demonstrate Rational Probabilistic Beliefs
Artificial Intelligence
Makes AI understand "maybe" better for trust.