UserSimCRS v2: Simulation-Based Evaluation for Conversational Recommender Systems
By: Nolwenn Bernard, Krisztian Balog
Potential Business Impact:
Builds better chatbots that suggest things you like.
Resources for simulation-based evaluation of conversational recommender systems (CRSs) are scarce. The UserSimCRS toolkit was introduced to address this gap. In this work, we present UserSimCRS v2, a significant upgrade aligning the toolkit with state-of-the-art research. Key extensions include an enhanced agenda-based user simulator, introduction of large language model-based simulators, integration for a wider range of CRSs and datasets, and new LLM-as-a-judge evaluation utilities. We demonstrate these extensions in a case study.
Similar Papers
Limitations of Current Evaluation Practices for Conversational Recommender Systems and the Potential of User Simulation
Information Retrieval
Makes chatbots recommend things better.
Search-Based Interaction For Conversation Recommendation via Generative Reward Model Based Simulated User
Information Retrieval
Teaches computers to guess what you like better.
Towards Personalized Conversational Sales Agents: Contextual User Profiling for Strategic Action
Information Retrieval
Helps shoppers buy things by talking to them.