Tag: Artificial Intelligence

Categories
Educational Administration and Leadership Higher Education

A comparative study of learning effects from human-made and AI-generated teaching videos

In the age of generative artificial intelligence (AI), can generative AI-made teaching videos deliver learning outcomes comparable to those of human-made videos? A recent study explored this question by comparing the effectiveness of four generative AI-made videos with four human-made videos on the same topics in a management course. The online intervention involved 447 US-based laypeople, most of whom had completed higher education. The study included two experimental conditions: in the first, 213 participants watched four teaching videos in a human-AI-human-AI sequence; in the second, 234 participants watched them in an AI-human-AI-human sequence. After each video, participants completed a survey rating their learning experience, followed by a multiple-choice exam to assess learning outcomes.

Results showed that human-made videos provided a small but statistically significant advantage in learning experience, suggesting that the participants still preferred human teachers in the videos. Regarding learning outcomes, participants in both conditions achieved a similar degree of acquired knowledge. The authors noted that as generative AI-made teaching videos become increasingly common due to their ease of production, researchers and educators should critically explore ways to use generative AI-made content to improve learning experiences, while being mindful of potential limitations and ethical considerations in production and use.

 

Source (Open Access): Netland, T., von Dzengelevski, O., Tesch, K., & Kwasnitschka, D. (2025). Comparing human-made and AI-generated teaching videos: An experimental study on learning effects. Computers & Education, 224, 105164. https://doi.org/10.1016/j.compedu.2024.105164… Read the rest

Categories
Educational Administration and Leadership Higher Education Social and Motivational Outcomes

Low academic self-efficacy may lead to AI dependency through stress

Generative AI tools are emerging in classrooms and on student laptops across the globe. Policymakers, educators, and other influencers must understand the negative consequences of becoming unhealthily dependent on AI. Using the Interaction of Person-Affect-Cognition-Execution (I-PACE) model, which examines how psychological factors contribute to problematic technology use, researchers studied AI dependency among 300 university students in Seoul. While they hypothesized a link between self-efficacy (students’ belief in their abilities) and AI dependency, they found no direct association. Instead, academic stress emerged as the key driver in AI dependency among students. In other words, the more academically stressed a student is, the more likely they are to become dependent on AI tools.

Notably, 84% of the students surveyed used ChatGPT for academic help, though not all showed signs of dependency. The researchers also sought to discover the negative consequences of AI dependency. Students reported that AI dependency led to increased laziness, the spread of misinformation, a lower level of creativity, and reduced critical and independent thinking. The researchers also examined the role of performance expectations: students who perceived that AI would help their performance were more likely to become dependent.

The emergency of AI tools in schools is a conundrum that generates many different opinions and policy recommendations. Rather than focusing primarily on AI restrictions and teaching AI literacy, schools may need to prioritize stress management and low-stakes practice opportunities to reduce AI dependency.

 

Source (Open Access): Zhang, S., Zhao, X., Zhou, T., & Kim, J. H. (2024). Do you have AI dependency? The roles of academic self-efficacy, academic stress, and performance expectations on problematic AI usage behavior. International Journal of Educational Technology in Higher Education, 21(1), 34. https://doi.org/10.1186/s41239-024-00467-0… Read the rest

Categories
Effective Teaching Approach K-12 Education Maths and Science Learning

Empowering tutoring expertise with AI

Effective tutoring can significantly improve student learning outcomes, but many students, particularly in under-served communities, often lack access to high-quality, expert-guided instruction due to resource limitations and the scarcity of trained educators. Stanford University researchers conducted the first randomized controlled trial of Tutor CoPilot, a Human-AI system designed to provide real-time, expert-like guidance to K-12 tutors, to explore its impact on enhancing tutor effectiveness during live sessions. In collaboration with FEV Tutor and a U.S. Southern school district, the researchers conducted an intervention involving 900 tutors and 1,800 students from Title I schools participating in an in-school, virtual tutoring program focused on mathematics.

The study showed that students whose tutors used Tutor CoPilot were 4 percentage points more likely to master mathematical lesson topics compared to those in the control group. This effect was especially pronounced among students taught by lower-rated tutors, whose mastery improved by 9 percentage points. The system also promoted the use of expert teaching strategies, such as prompting students to explain their reasoning and asking guiding questions, rather than giving away answers, fostering deeper student understanding. Despite some challenges, such as occasional misalignment of AI suggestions with student grade levels, tutors reported that Tutor CoPilot helped them better address student needs. With an annual cost of just $20 per tutor, Tutor CoPilot offers a scalable and affordable path to improving tutoring quality in contexts where expert educators are in short supply. This study illustrates the potential of Human-AI systems like Tutor CoPilot to make high-quality learning accessible to all students.

 

Source (Open Access): Wang, Rose E., Ribeiro, Ana T., Robinson, Carly D., Loeb, Susanna, & Demszky, Dorottya. (2024). Tutor copilot: A human-AI approach for scaling real-time expertise. (EdWorkingPaper: 24 -1056). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/81NH-8262… Read the rest

Categories
Language Development Maths and Science Learning Secondary School Education

Assessing the quality of feedback from humans and ChatGPT on students’ writing

Automated writing evaluation (AWE) tools aid educators by quickly evaluating student writing, but often at the expense of accuracy and clarity, and setting them up requires substantial effort and resources. In contrast, generative AI like ChatGPT provides timely, targeted, and adaptive feedback that can enhance student writing. A recent study compared the quality of feedback from ChatGPT and human raters, with potential implications for process-based writing evaluation.

The study involved 200 students randomly sampled across grades 6-12 from two school districts in Southern California. Over two 50-minute class periods, students were tasked with responding to one of two writing prompts. Each prompt demanded that they read four primary and secondary sources and craft an argument analyzing historical interpretations using evidence, reasoning, and required writing skills. Sixteen experienced secondary educators from various disciplines were recruited and trained to provide formative feedback. This feedback was then compared to ChatGPT’s feedback, which was prompted to reflect the same context used by the educators. The researchers coded and analyzed the feedback for comparison.

The findings revealed that well-trained, paid, and relatively time-rich human raters offered higher quality feedback in four of five key areas: clarity of improvement directions, accuracy, prioritization of essential features, and use of a supportive tone (the fifth area: criteria-based feedback). However, ChatGPT’s writing feedback closely matched human feedback in quality, without requiring any training, and the differences in quality between ChatGPT and human feedback were modest, considering both overall quality and time savings. The researchers concluded that generative AI could be a useful tool in specific situations, especially for formative drafts or when highly trained educators are unavailable.

 

Source (Open Access): Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024). Comparing the quality of human and ChatGPT feedback of students’ writing. Learning and Instruction, 91, 101894. https://doi.org/10.1016/j.learninstruc.2024.101894… Read the rest

Categories
Maths and Science Learning Programme Evaluation Secondary School Education

Generative AI’s impact on student learning: A double-edged sword

Generative AI tools, like OpenAI’s GPT-4, are increasingly integrated into educational settings, promising enhanced learning and productivity. However, their long-term impact on skill acquisition remains under scrutiny. A recent research paper from the Wharton School of the University of Pennsylvania details a large-scale randomized controlled trial (RCT) designed to assess how generative AI affects student learning, focusing specifically on math classes in a high school setting. The study involved nearly 1,000 students across three grades, evaluating two GPT-4-based tutors: GPT Base and GPT Tutor. The GPT Base provided a standard ChatGPT interface, while GPT Tutor incorporated safeguards to support learning without providing direct answers.

The results revealed that while GPT-4 significantly improved immediate performance on practice problems—by 48% for GPT Base and 127% for GPT Tutor—these gains did not translate into long-term learning. When access to GPT-4 was removed during exams, students who had used GPT Base performed 17% worse than those who never had access, indicating a detrimental effect on learning. However, GPT Tutor mitigated this negative impact, with performance differences becoming statistically insignificant.

The study underscores the potential of generative AI to enhance short-term performance but also highlights the risk of overreliance on these tools, which can inhibit the development of essential problem-solving skills. As educational institutions increasingly turn to AI-driven tools, the findings stress the need for carefully designed safeguards to ensure that students continue to learn and retain critical skills over time.

 

Source (Open Access): Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2024). Generative AI can harm learning. The Wharton School Research Paper. https://www.ssrn.com/abstract=4895486… Read the rest

Categories
Educational Administration and Leadership K-12 Education Language Development

Chatbot interaction shows positive effects on language learning

Chatbots are reminiscent of a gossipy co-worker—full of chatter, not all of it credible, yet nonetheless useful now and then. In a meta-analysis of 28 studies comparing Chatbot users to non-users, Wang, Cheung, Neitzel, & Chai recently examined the effects of chatbots on users’ language learning, and the moderators that affected this interaction in terms of 18 characteristics of the humans and chatbots.

Results showed that chatbot users demonstrated greater gains in language learning than comparable non-users (ES=+0.48). Other findings included that language gains were greatest for secondary school students and adult learners; second language learners at the intermediate level (versus beginning or advanced), particularly those learning English; and for those who used a mobile-based program instead of a laptop or PC. Chatbot-driven interaction showed greater gains than user-driven interactions, and were most effective when used 1:1 in-class instead of out-of-class. Voice-based and voice-plus-text-based interactions both demonstrated positive effects on language learning. Implications for the classroom are that chatbots are a valid in-class language learning tool for older students, especially those learning English as a second language.

 

Source: Wang, F., Cheung, A. C. K., Neitzel, A. J., & Chai, C. S. (2024). Does chatting with chatbots improve language learning performance? A meta-analysis of chatbot-assisted language learning. Review of Educational Research, 00346543241255621. https://doi.org/10.3102/00346543241255621… Read the rest