Factors affecting the in-context learning abilities of LLMs for dialogue state tracking
By: Pradyoth Hegde , Santosh Kesiraju , Jan Švec and more
Potential Business Impact:
Helps computers understand what you're saying in chats.
This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking.
Similar Papers
Beyond Single-User Dialogue: Assessing Multi-User Dialogue State Tracking Capabilities of Large Language Models
Computation and Language
Makes computers understand many people talking at once.
Interpretable and Robust Dialogue State Tracking via Natural Language Summarization with LLMs
Computation and Language
Helps chatbots understand what you're saying better.
Leveraging In-Context Learning for Language Model Agents
Computation and Language
Helps AI agents learn by watching examples.