While AI-powered learning technologies are increasingly used to automate and support learning activities, often with positive outcomes, their impact on student agency is under-explored. Student agency refers to students’ capacity to actively regulate learning actions, make responsible decisions, and navigate various learning contexts, which is essential for lifelong learning. A recent randomized controlled experiment explored the impact of AI assistance on student agency in higher education, addressing three research questions: Do students learn from AI assistance? After an initial period of time, can AI assistance be replaced with self-monitoring checklists? Would complementing AI assistance with self-monitoring checklists enhance student performance?
The study involved 1625 undergraduate students across 10 courses from various disciplines in 2020. During the initial four-week period, students provided peer-reviewed comments to each other, guided by AI features to enhance their feedback. Over the following four-week period, they were divided into four groups: a non-AI-assisted group, an AI-assisted group, a self-monitoring group without AI assistance, and a self-monitoring group with AI assistance. The study used six measures to evaluate student agency from different perspectives of students’ reviews: rate of reviews that needed revision, similarity to previous comments, relatedness to reviewed resources, review length, time spent on reviews, and helpfulness ratings from other reviewers.
Results showed that AI assistance significantly improved the quality of students’ reviews, but the influence declined after AI assistance was removed. This suggests that while AI can effectively scaffold learning, students tend to rely on it rather than learn from it. Additionally, after using AI assistance for some time, students can still benefit from self-monitoring checklists even without AI assistance. However, combining AI assistance with self-regulation strategies did not lead to significant improvement in student performance. The authors attribute the insignificant improvement to two possible reasons. First, when supports of varying strengths interact, the stronger one may overshadow or diminish the impact of the weaker one. Second, learners have limited cognitive resources, which can be overwhelmed if the cognitive load exceeds their capacity, so the higher load from AI assistance might have reduced their capacity to effectively use self-monitoring checklists. The authors concluded that while AI-powered learning technologies present many benefits, they should be used with caution, taking into account pedagogical factors and meticulously balancing potential benefits against possible drawbacks.
Source (Open Access): Darvishi, A., Khosravi, H., Sadiq, S., Gašević, D., & Siemens, G. (2024). Impact of AI assistance on student agency. Computers & Education, 210, 104967. https://doi.org/10.1016/j.compedu.2023.104967