Score: 3

UserSimCRS v2: Simulation-Based Evaluation for Conversational Recommender Systems

Published: December 4, 2025 | arXiv ID: 2512.04588v1

By: Nolwenn Bernard, Krisztian Balog

Potential Business Impact:

Builds better chatbots that suggest things you like.

Business Areas:
Simulation Software

Resources for simulation-based evaluation of conversational recommender systems (CRSs) are scarce. The UserSimCRS toolkit was introduced to address this gap. In this work, we present UserSimCRS v2, a significant upgrade aligning the toolkit with state-of-the-art research. Key extensions include an enhanced agenda-based user simulator, introduction of large language model-based simulators, integration for a wider range of CRSs and datasets, and new LLM-as-a-judge evaluation utilities. We demonstrate these extensions in a case study.

Country of Origin
🇳🇴 🇩🇪 Germany, Norway

Repos / Data Links

Page Count
16 pages

Category
Computer Science:
Information Retrieval