Examining Student Interactions with a Pedagogical AI-Assistant for Essay Writing and their Impact on Students Writing Quality
By: Wicaksono Febriantoro , Qi Zhou , Wannapon Suraworachet and more
The dynamic nature of interactions between students and GenAI, as well as their relationship to writing quality, remains underexplored. While most research has examined how general-purpose GenAI can support writing, fewer studies have investigated how students interact with pedagogically designed systems across different phases of the writing process. To address this gap, we evaluated a GenAI-driven essay-writing assistant (EWA) designed to support higher education students in argumentative writing. Drawing on 1,282 interaction logs from 32 undergraduates during a two-hour writing session, Sequential Pattern Mining and K-Means clustering were used to identify behavioral patterns. Two clusters emerged: Cluster 1 emphasized outline planning and essay structure, while Cluster 2 focused on content development. A Mann-Whitney U test revealed a moderate effect size (r = 0.36) in the essay Organization dimension, with Cluster 1 showing higher scores. Qualitative analysis indicated that students with better performance actively wrote and shared essay sections with EWA for feedback, rather than interacted passively by asking questions. These findings suggest implications for teaching and system design. Teachers can encourage active engagement, while future EWAs may integrate automatic labeling and monitoring to prompt students to move from questioning to writing, enabling fuller benefits from GenAI-supported learning.
Similar Papers
Writing With Machines and Peers: Designing for Critical Engagement with Generative AI
Computers and Society
Teaches students to use AI for better writing.
Approaching the Limits to EFL Writing Enhancement with AI-generated Text and Diverse Learners
Computation and Language
AI helps students write better, but only some.
How Problematic Writer-AI Interactions (Rather than Problematic AI) Hinder Writers' Idea Generation
Human-Computer Interaction
AI helps students learn better when they think.