Manifesto from Dagstuhl Perspectives Workshop 24352 -- Conversational Agents: A Framework for Evaluation (CAFE)
By: Christine Bauer , Li Chen , Nicola Ferro and more
Potential Business Impact:
Helps computers understand and answer questions better.
During the workshop, we deeply discussed what CONversational Information ACcess (CONIAC) is and its unique features, proposing a world model abstracting it, and defined the Conversational Agents Framework for Evaluation (CAFE) for the evaluation of CONIAC systems, consisting of six major components: 1) goals of the system's stakeholders, 2) user tasks to be studied in the evaluation, 3) aspects of the users carrying out the tasks, 4) evaluation criteria to be considered, 5) evaluation methodology to be applied, and 6) measures for the quantitative criteria chosen.
Similar Papers
CAF-I: A Collaborative Multi-Agent Framework for Enhanced Irony Detection with Large Language Models
Computation and Language
Helps computers understand sarcasm better.
ISCA: A Framework for Interview-Style Conversational Agents
Computation and Language
Builds smart chatbots for interviews.
Convivial Conversational Agents -- shifting toward relationships
Human-Computer Interaction
Helps AI talk to people with memory loss.