Tag: Artificial Intelligence

Categories
Higher Education

The Role of Undergraduates’ Critical Thinking in Generative AI Reliance Behaviors

Hou and colleagues conducted a large-scale survey study using structural equation modeling (SEM) to examine how undergraduates’ critical thinking influences their different types of reliance on generative AI during problem-solving tasks. The study analyzed 808 valid responses, measuring students’ critical thinking skills and dispositions, AI literacy, trust in AI, and four types of AI-use behaviors—reflective, cautious, collaborative, and thoughtless use. The authors conceptualized reliance behavior as the way learners evaluate and make use of the differences between AI and human abilities, and proposed that critical thinking may play a key moderating role in this process.

The results showed that AI literacy strongly predicted both critical thinking skills (β = .66, p < .001) and dispositions (β = .41, p < .001), whereas trust in AI was negatively related to both (skills: β = –.16, p < .05; disposition: β = –.11, p < .001). Regarding reliance behaviors, critical thinking skills were positively associated with collaborative use (β = .25), reflective use (β = .21), and cautious use (β = .24), with similar effects found for critical thinking disposition. These findings highlight the importance of critical thinking in supporting desirable forms of AI use. In contrast, trust strongly predicted thoughtless use (β = .47, p < .001) and also slightly increased collaborative use (β = .15, p < .05) and reflective use (β = .19, p < .001), indicating a dual role of trust in both strengthening and weakening ideal reliance behaviors. More importantly, AI literacy promoted collaborative (β = .25), reflective (β = .20), and cautious use (β = .22) through the mediation of critical thinking, whereas trust produced negative indirect effects on these desirable behaviors because it reduced critical thinking (β = –.05 to –.06, p < .001). This means that critical thinking both enhances the positive influence of AI literacy and suppresses the potential blind reliance brought by high trust, guiding learners toward more reflective, careful, and collaborative ways of using AI.

Overall, the study provides strong evidence that critical thinking does not simply reduce AI reliance; instead, it shapes how students rely on AI, encouraging forms of use that are more reflective, collaborative, and prudent. The authors argue that the development of AI literacy must be accompanied by the cultivation of critical thinking to reduce thoughtless dependence and to promote healthier human–AI collaboration. They also emphasize that educational interventions should clearly define “ideal reliance behaviors” and help students develop responsible and thoughtful habits of AI use in an era where generative AI is becoming increasingly widespread.

Source (Open Access): Hou, C., Zhu, G., & Sudarshan, V. (2025). The role of critical thinking on undergraduates’ reliance behaviours on generative AI in problem‐solving. British Journal of Educational Technology, 56(5), 1919-1941.

https://doi.org/10.1111/bjet.13613… Read the rest

Categories
Higher Education Social and Motivational Outcomes

Learners’ Preferences for Feedback from AI and Human Instructors

Le and his team examined whether learners’ preferences for feedback from human instructors versus generative artificial intelligence (AI) would change after receiving feedback from different sources and interface types in an academic English writing task. The study recruited 114 university students who were non-native English speakers and randomly assigned them to four groups: no feedback (control), human instructor feedback, ChatGPT 4.0 in a free-conversation interface, and a structured writing analysis tool powered by ChatGPT. Learners’ preferences were measured both before and after the task using rating scales and binary-choice questions, and the four groups were compared in terms of post-task preference and preference change.

The results showed that learners already had a clear preference for human instructors before the task (87.2% chose human), and this preference remained stable after the task (86.0% chose human), reflecting a phenomenon of algorithm aversion in educational settings. However, post-test preference scores differed significantly among the four groups: the human instructor group rated significantly higher than both the free-conversation AI group and the control group. On the binary human/AI choice measure, significant differences were also found — the human instructor and structured AI tool groups both scored higher than the free-conversation AI group. Regarding preference change, the overall mean shift was close to zero, but the differences among groups were significant: the free-conversation AI group showed a slight increase in preference for AI, whereas the human instructor and structured AI tool groups remained more favorable toward humans. In other words, although all three feedback types were effective, the free-conversation interface was the only one that reduced algorithm aversion and increased learners’ acceptance of AI, while the structured, one-time feedback tool further reinforced their preference for human instructors.

Based on these findings, the authors argue that enhancing the interactivity and dialogic nature of AI-based learning tools may influence learners’ preferences more effectively than purely improving their technical performance. Interactive dialogue allows for clarification and correction, which reduces learners’ unrealistic expectations that algorithms must be perfect and mitigates distrust. Overall, the study situates human preference within the context of interface design, providing both empirical insights and cautions for the adoption, product design, and pedagogical integration of AI in education.

 

Source (Open Access): Le, H., Shen, Y., Li, Z., Xia, M., Tang, L., Li, X., … & Fan, Y. (2025). Breaking human dominance: Investigating learners’ preferences for learning feedback from generative AI and human tutors. British Journal of Educational Technology.

https://doi.org/10.1111/bjet.13614… Read the rest

Categories
Educational Administration and Leadership Effective Teaching Approach Secondary School Education

Leveraging AI to predict young learners’ online learning engagement

With many schools rushing to adopt Generative AI, it is important to consider the real learning gains (or lack thereof) that these tools offer. A 2023 study by Pardos & Bhandari examined the use of AI-generated hints as a scaffolding mechanism with Algebra students.

Seventy-seven participants (high school graduates selected via Amazon’s MTURK system) were assigned to a control group (which provided human-generated hints) or an experimental group (which provided AI-generated hints). The researchers wanted to learn the rate of “low quality” AI-generated hints, as well as if the hints produced learning gains compared to the control group. The questions from the lesson were fed, verbatim, to ChatGPT in order to generate the hints. Quality checks were performed manually to ensure that all AI-generated hints were correct and showed the proper steps. This was then contrasted with the control group, whose hints were generated by undergraduate tutors. Pre and post tests were administered to check for learning gains between the two groups.

The results showed that 70% of the hints generated by ChatGPT were considered to be good quality, and that there was a statistically significant learning gain in the control group. A major limitation of the study is that the researchers did not prompt the AI to use any scaffolding strategies. Therefore, the quality of the hints between groups not only differed by human or AI creator, but also by pedagogical theory. Human tutors were probably more likely to employ Vygotsky-esque scaffolds, while ChatGPT was more likely to provide an immediate answer. Future work could improve upon the prompts used in this study and create a multi-tiered approach with less consequential hints being revealed at first.

 

Source (Open Access): Pardos, Z. A., & Bhandari, S. (2023). Learning gain differences between ChatGPT and human tutor generated algebra hints (No. arXiv:2302.06871). https://doi.org/10.48550/arXiv.2302.06871… Read the rest

Categories
Educational Administration and Leadership Higher Education

Metacognitive support for self-regulated learning in GenAI environment

A recent study by Xu and colleagues investigated the impact of metacognitive support on self-regulated learning (SRL) and learning experiences within generative AI (GenAI) environments. The study addressed concerns about students’ reliance on ChatGPT and the challenges they face in sustaining SRL without guidance.

Participants included 68 sophomore students in a university in China, aged 19–21, majoring in Educational Technology, who had no prior experience with GenAI tools in classroom tasks. The students were randomly divided into two groups: the experimental group received a metacognitive support framework, while the control group did not. Delivered on paper to reduce distractions, the framework encouraged planning, monitoring, and reflecting during tasks, with prompts like setting clear goals, evaluating ChatGPT’s feedback, and reflecting on task performance. Over a four-week period, the participants completed interdisciplinary tasks that integrated knowledge from multiple subjects.

Based on self-report questionnaires, the experimental group demonstrated significant improvements in two of six SRL abilities: in task strategies (ES = +0.69) and self-evaluation (ES = +0.53), while the control group exhibited declines in these areas. Students with metacognitive support also reported lower cognitive load and perceived AI tools as more useful, although these findings were not statistically significant. Academic performance gains were higher in the experimental group (ES = +0.36) but the difference was not statistically significant.

This study suggests that metacognitive support helps students engage with AI tools and develop active learning. Without such guidance, students experience a decline in self-regulation skills which could undermine learning autonomy.

 

Source: Xu, X., Qiao, L., Cheng, N., Liu, H., & Zhao, W. (2025). Enhancing self-regulated learning and learning experience in generative AI environments: The critical role of metacognitive support. British Journal of Educational Technology, 0(0). https://doi.org/10.1111/bjet.13599… Read the rest

Categories
Educational Administration and Leadership Secondary School Education

Can AI reduce teacher workload? Early evidence from a UK trial with ChatGPT

Generative Artificial Intelligence (GenAI) tools like ChatGPT are becoming increasingly common in classrooms—not just for students, but also for teachers. In England, the Department for Education has acknowledged that educators are using GenAI more often to plan lessons, create teaching materials, and even write exam questions. A major reported advantage is the potential to save time, which is especially relevant as workload remains a key factor behind teacher attrition.

To explore whether AI can help reduce this burden, the National Foundation for Educational Research recently conducted a rigorous trial. The study involved 68 secondary schools and 259 science teachers, who were randomly assigned to prepare Year 7 and 8 science lessons either with or without ChatGPT. Teachers in the ChatGPT group were given a practical guide to support their use of the tool. Over a 10-week period in the summer term of 2024, they logged how much time they spent preparing lessons, with a particular focus on weeks 6 to 10—after an initial adaptation phase.

The findings were encouraging. On average, teachers using ChatGPT spent 25 minutes less per week on lesson preparation than those in the non-AI group—56 minutes versus 81.5—representing a 31% time saving. Importantly, an independent expert panel found no difference in the quality of lesson materials between the two groups.

Use of the support guide also declined over time, suggesting that teachers grew more confident in integrating the tool into their practice. Looking ahead, future research could explore how GenAI tools like ChatGPT are used for other aspects of teachers’ work—such as administrative duties—and whether their impact differs across subjects or age groups, especially as new and more advanced versions continue to roll out.

 

Source (Open Access): Roy, P., Poet, H., Staunton, R., Aston, K., & Thomas, D. (2024). ChatGPT in lesson preparation—A Teacher Choices Trial. National Foundation for Educational Research. https://www.isrctn.com/ISRCTN13420346… Read the rest

Categories
Higher Education Language Development

Will GenAI Chatbot Enhance Self-Regulated Learning and Reading Engagement for EFL Students?

A quasi-experiment by Pan and colleagues explored whether a generated AI chatbot, Reade, could enhance self-regulated learning (SRL) in reading strategy and reading engagement among English as a foreign language (EFL) students. The study involved 61 first-year English major students (over 80% female, mean age = 18.5) at a university in eastern China. Two classes were randomly assigned to a treatment (n=31) or a control group (n=30).

Using the research team-developed ReadMate platform, powered by ChatGPT 3.5 Turbo, students accessed Reade, which had two main functions: (1) recommending reading materials based on student proficiency and preferences, and (2) supporting self-regulated reading strategies through a Plan-Enact-Reflect approach. At the “Enact” phase, the treatment group students had access to full functions in ReadMate, including GenAI-empowered SRL interaction for further guidance on SRL strategies, while the control group used ReadMate without this feature. The 12-week study required participants to complete reading tasks via ReadMate as part of their coursework. Questionnaires reported self-regulated reading strategy use and reading engagement before and three weeks after the intervention.

After controlling for baseline scores, the results revealed that the treatment group significantly outperformed the control group in SRL strategy use across metacognitive, cognitive, motivational, and behavioral dimensions. Platform logs indicated that the treatment group accessed Planning Strategies and Self-Checklist panels more frequently. Additionally, the treatment group exhibited higher reading engagement across cognitive, behavioral, and emotional dimensions, as well as greater time and effort invested in reading tasks on the platform.

Although the study highlights the potential of GenAI chatbots like Reade in supporting SRL and reading engagement, its reliance on a small sample and self-reported data underscores the need for more rigorous research to explore AI’s role in enhancing EFL students’ reading outcomes.

 

Source: Pan, M., Lai, C., & Guo, K. (2025). Effects of GenAI-empowered interactive support on university EFL students’ self-regulated strategy use and engagement in reading. The Internet and Higher Education, 65, 100991. https://doi.org/10.1016/j.iheduc.2024.100991… Read the rest