LLM-as-a-Judge: Toward World Models for Slate Recommendation Systems
By: Baptiste Bonin, Maxime Heuillet, Audrey Durand
Potential Business Impact:
Helps computers guess what you want to see next.
Modeling user preferences across domains remains a key challenge in slate recommendation (i.e. recommending an ordered sequence of items) research. We investigate how Large Language Models (LLM) can effectively act as world models of user preferences through pairwise reasoning over slates. We conduct an empirical study involving several LLMs on three tasks spanning different datasets. Our results reveal relationships between task performance and properties of the preference function captured by LLMs, hinting towards areas for improvement and highlighting the potential of LLMs as world models in recommender systems.
Similar Papers
The Application of Large Language Models in Recommendation Systems
Information Retrieval
Makes online suggestions smarter and more personal.
Do LLM-judges Align with Human Relevance in Cranfield-style Recommender Evaluation?
Information Retrieval
Lets computers judge movie recommendations fairly.
LLM as Explainable Re-Ranker for Recommendation System
Information Retrieval
Helps online stores show you better, clearer choices.