Methodological Foundations for AI-Driven Survey Question Generation
By: Ted K. Mburu , Kangxuan Rong , Campbell J. McColley and more
Potential Business Impact:
Makes smart computer questions for school surveys.
This paper presents a methodological framework for using generative AI in educational survey research. We explore how Large Language Models (LLMs) can generate adaptive, context-aware survey questions and introduce the Synthetic Question-Response Analysis (SQRA) framework, which enables iterative testing and refinement of AI-generated prompts prior to deployment with human participants. Guided by Activity Theory, we analyze how AI tools mediate participant engagement and learning, and we examine ethical issues such as bias, privacy, and transparency. Through sentiment, lexical, and structural analyses of both AI-to-AI and AI-to-human survey interactions, we evaluate the alignment and effectiveness of these questions. Our findings highlight the promise and limitations of AI-driven survey instruments, emphasizing the need for robust prompt engineering and validation to support trustworthy, scalable, and contextually relevant data collection in engineering education.
Similar Papers
Generative AI as a Safety Net for Survey Question Refinement
Methodology
Helps make survey questions clear and accurate.
Designing LMS and Instructional Strategies for Integrating Generative-Conversational AI
Computation and Language
AI helps teachers give every student special lessons.
Harnessing the Power of AI in Qualitative Research: Role Assignment, Engagement, and User Perceptions of AI-Generated Follow-Up Questions in Semi-Structured Interviews
Human-Computer Interaction
AI helps interviewers ask better questions.