We Should Evaluate Real-World Impact
By: Ehud Reiter
Potential Business Impact:
Helps computers understand language better in real life.
The ACL community has very little interest in evaluating the real-world impact of NLP systems. A structured survey of the ACL Anthology shows that perhaps 0.1% of its papers contain such evaluations; furthermore most papers which include impact evaluations present them very sketchily and instead focus on metric evaluations. NLP technology would be more useful and more quickly adopted if we seriously tried to understand and evaluate its real-world impact.
Similar Papers
Good Intentions Beyond ACL: Who Does NLP for Social Good, and Where?
Computation and Language
Helps computers help people with world problems.
Internal and External Impacts of Natural Language Processing Papers
Computation and Language
Shows how computer language skills are used.
Has ACL Lost Its Crown? A Decade-Long Quantitative Analysis of Scale and Impact Across Leading AI Conferences
Digital Libraries
Shows how AI conferences are changing over time.