Using a randomized controlled experimental design, Shi et al. (2025) compared the effects of ChatGPT-based feedback and traditional automated writing evaluation (AWE) systems on English-as-a-foreign-language (EFL) students’ writing performance and their ideal L2 writing self. One hundred and fifty second-year university students from three writing classes in a Chinese public university were recruited and randomly divided into a ChatGPT group, an AWE group, and a control group.
After an eleven-week intervention, results showed that ChatGPT helped students perform better in their writing compared to the control group and the AWE group, but compared to the AWE group, ChatGPT significantly lowered students’ ideal L2 writing self. Qualitative results shed light on possible causes: while participants were fully aware of the affordances of ChatGPT feedback, they were also concerned with their (over) reliance on the tool and the accompanying loss of creativity and agency and expressed their reserved attitude toward future intention to use ChatGPT.
Educators should refine learning objectives based on students’ ZPD and design prompts accordingly, so that ChatGPT supports learning rather than completing tasks, while also teaching prompt-engineering skills. For lower-intermediate to intermediate learners, AWE’s systematic and rule-based feedback can provide stronger scaffolding and better preserve authorship. However, ChatGPT’s richer affordances may lead to over-reliance, weakening learner agency and diminishing the ideal L2 writing self. Therefore, language-education goals should be redefined to incorporate AI literacy and critical thinking, safeguarding teacher and learner agency and promoting responsible use.
Source (Open Access): Shi, H., Chai, C. S., Zhou, S., & Aubrey, S. (2025). Comparing the effects of ChatGPT and automated writing evaluation on students’ writing and ideal L2 writing self. Computer Assisted Language Learning, 1-28.