Category: Educational Administration and Leadership

Categories
Educational Administration and Leadership K-12 Education Language Development

Chatbot interaction shows positive effects on language learning

Chatbots are reminiscent of a gossipy co-worker—full of chatter, not all of it credible, yet nonetheless useful now and then. In a meta-analysis of 28 studies comparing Chatbot users to non-users, Wang, Cheung, Neitzel, & Chai recently examined the effects of chatbots on users’ language learning, and the moderators that affected this interaction in terms of 18 characteristics of the humans and chatbots.

Results showed that chatbot users demonstrated greater gains in language learning than comparable non-users (ES=+0.48). Other findings included that language gains were greatest for secondary school students and adult learners; second language learners at the intermediate level (versus beginning or advanced), particularly those learning English; and for those who used a mobile-based program instead of a laptop or PC. Chatbot-driven interaction showed greater gains than user-driven interactions, and were most effective when used 1:1 in-class instead of out-of-class. Voice-based and voice-plus-text-based interactions both demonstrated positive effects on language learning. Implications for the classroom are that chatbots are a valid in-class language learning tool for older students, especially those learning English as a second language.

 

Source: Wang, F., Cheung, A. C. K., Neitzel, A. J., & Chai, C. S. (2024). Does chatting with chatbots improve language learning performance? A meta-analysis of chatbot-assisted language learning. Review of Educational Research, 00346543241255621. https://doi.org/10.3102/00346543241255621… Read the rest

Categories
Educational Administration and Leadership Kindergarten Programme Evaluation

Building parent-teacher connections in preschool

Inspiring and supporting parent-teacher relationships and family involvement in a child’s education starting in preschool has the potential to instill stronger academic engagement at a foundational level. This is the theory of case for Purtell and colleagues, who examined the impact of the Kindergarten Transition Practices (KTP) intervention on parental engagement, with particular focus on how impacts varied by family race/ethnicity, maternal education, and children’s behavior problems.

In their study, 391 students were split between a classroom-level intervention (KTP-Classroom), a classroom intervention coupled with home visits (KTP-Plus), and a business-as-usual (BAU) control group. KTP is designed to build connections and relationships through a variety of practices, with a particular focus on families facing economic disadvantages and other structural barriers that may impede strong parent-school connections. KTP-Classroom includes school-based events throughout the year for parents and children, monthly newsletters, and meetings focused on preschooler transition to kindergarten. Teachers are supported by “transition coordinators,” to help design and implement these activities. KTP-Plus included nine home visits, during preschool, summer, and kindergarten, conducted by transition coordinators and focused on engaging parents in their children’s schooling and building teacher/school-parent connections and developing ecologically informed practices.

Based on teacher surveys of parental engagement, Purtell’s randomized study found significantly higher teacher-reported parent involvement among KTP-Classroom families, though no significant effect was seen on parent-teacher relationships or parent values. In addition, researchers found the classroom plus home visit (KTP-Plus) intervention led to more positive teacher perceptions of parent involvement, parent-teacher relationships, and parent values among Hispanic families.

Teacher perceptions of improved engagement and values are important indicators of progress, but it should be noted that this study does not include parental perspective change or measures of student academic achievement or engagement.

 

Source: Purtell, K. M., Jiang, H., Justice, L. M., Sayers, R., Dore, R., & Pelfrey, L. (2022). Teacher perceptions of preschool parent engagement: Causal effects of a connection-focused intervention. Child & Youth Care Forum, 51(5), 937–966. https://doi.org/10.1007/s10566-021-09661-x… Read the rest

Categories
Educational Administration and Leadership Maths and Science Learning Primary School Education

Is time spent on extracurricular physical activity detrimental math performance?

Although the benefits of physical activity for children are well-documented, concerns persist among parents and school administrators that more time devoted to physical activities could reduce study time and hinder academic performance. To address these concerns, Wang and colleagues conducted a cluster randomized clinical trial exploring the impact of extracurricular after-school physical activities on math performance, physical fitness, and myopia status among 3rd and 4th graders in Jiangxi, China.

The study involved 24 elementary schools, stratified into six strata, based on location and prior math scores. Schools within each stratum were randomly assigned to either the intervention group or the control group. The intervention group (12 schools, n = 1012) participated in a 2-hour extracurricular physical activity program after school on school days, engaging in activities such as basketball, soccer, badminton, and table tennis. The control group (12 schools, n = 1020) had no structured after-school physical activity program and students managed their own after-school time.

Data collected before the intervention (October 2020) and after (June 2021) included math test scores, physical fitness scores, myopia status, and self-reported behavior-related data. The findings revealed:

  • Math performance in the intervention group was not inferior to the control group.
  • Students in the intervention group showed significantly greater improvements in overall physical fitness and a lower incidence of new myopia cases, although the latter was not statistically significant.
  • There was a lower proportion of students in the intervention group who spent more than 30 minutes per day on screen time.
  • The time spent on studying and reading was similar between both groups.

These results suggest that participating in after-school extracurricular physical activities for one academic year not only enhances physical fitness but also does not negatively impact math performance. Furthermore, it may offer additional benefits such as reducing excessive screen time.

 

Source: Wang, D., Xiong, R., Zhang, J., Han, X., Jin, L., Liu, W., Qu, Y., Chen, Q., Chen, S., Chen, X., Li, Y., He, M., Zeng, Y., & Liu, Y. (2023). Effect of extracurricular after-school physical activities on academic performance of schoolchildren: A cluster randomized clinical trial. JAMA Pediatrics, 177(11), 1141–1148. https://doi.org/10.1001/jamapediatrics.2023.3615… Read the rest

Categories
Educational Administration and Leadership K-12 Education Programme Evaluation

School dropout interventions: Assessing what works

School dropout is a problem of global concern due to the negative personal and social consequences it can lead to. Numerous intervention programs, targeting either all students or specifically those identified as at-risk, have been developed and implemented to reduce the dropout rate, although they have shown mixed results regarding their effectiveness. To investigate whether these inconsistent results are due to intervention characteristics and/or study methods, Wang and colleagues conducted a meta-analysis of 26 experimental or quasi-experimental studies related to school completion in K-12 US education.

The meta-analysis found a positive and statistically significant effect of dropout programs (ES= +0.19). Findings revealed that using randomly assigned methods or quasi-experimental matched methods produced a larger effect size than non-random methods. Moreover, compared to interventions that exclusively targeted academics (ES = +0.12), behavioral interventions (that address emotional and/or behavioral factors contributing to a student’s decision to leave school) and combined interventions produced greater effect sizes (respectively, ES = +0.33 and ES = +0.31). These results suggest integrating both behavioral and multicomponent interventions to tackle school dropout, and ensuring equivalent baseline participant characteristics between treatment and control groups in school dropout studies.

 

Source: Wang, Q., Hsiao, Y.-Y., Hushman, C., & Armstrong, J. (n.d.). The effectiveness of dropout intervention programs among K-12 students: A meta-analysis. Journal of Education for Students Placed at Risk (JESPAR), 0(0), 1–31. https://doi.org/10.1080/10824669.2024.2342779… Read the rest

Categories
Educational Administration and Leadership Kindergarten Programme Evaluation

Can the LfL PD program enhance teaching quality in Shenzhen kindergartens?

Professional development (PD) programs in China prioritize content knowledge transfer from teachers to students and tend to lack emphasis on enhancing teacher-child interaction. A cluster RCT study by Huang and colleagues investigated whether an adapted western PD intervention will improve both domain-specific instruction quality and teacher-child interaction strategies in Chinese kindergartens.

The LfL program covers diverse content knowledge, intentional and relational instruction, and the use of responsive teacher–child interactions to meet students’ needs. The adapted program included three PD phases: a 1-day intensive training on quality teaching, followed by six 3.5-hour sessions focusing on domain-specific content knowledge in literacy, math, science, self-regulation, diversity, and assessment. The final phase consisted of biweekly onsite mentoring from March to May 2020 to help teachers apply their new skills. All training was conducted face-to-face.

Stratified random sampling was used to select 24 kindergartens in Shenzhen, a developed area in China, dividing them into 12 control and 12 intervention schools comprising 95 classrooms 202 teachers and 547 children aged 3 to 6 years (K1 and K2). Teachers in the intervention group received the adapted LfL training, while those in the control group continued their regular teaching practice. The Early Childhood Environment Rating Scale-Extension (ECERS-E) measured content-related instruction quality and the Sustained Shared Thinking and Emotional Well-being (SSTEW) evaluated the interaction quality of supporting children’s socioemotional well-being and cognitive domain. Classroom observations were conducted once before (fall 2020) and once after the intervention (summer 2021). IDELA was used to assess children’s literacy, numeracy, socioemotional skills, and executive functions. Data collection was conducted by research assistants blind to participant status.

Multilevel analysis revealed that LfL-teachers outperformed control group in delivering knowledge (ECERS-E, ES = +0.81)) and interaction with students (SSTEW, ES = +0.72). Compared to the control group, children in intervention classrooms showed better performance in literacy (ES = +0.16) and executive function (ES = +0.27), but not in numeracy and socioemotional scores.

The authors suggest policymakers provide national PD programs based on an evidence-based approach for more effective resources’ allocation. An effective PD program could be cross-culturally and contextually adapted to strengthen education quality.

 

Source: Huang, R., Siraj, I., & Melhuish, E. (2024). Promoting effective teaching and learning through a professional development program: A randomized controlled trial. Journal of Educational Psychology. https://doi.org/10.1037/edu0000851… Read the rest

Categories
Educational Administration and Leadership Maths and Science Learning Secondary School Education

Effective and economical: Cost analysis of the ASSISTments online math program

With the growing interest in AI-supported and technology-focused interventions, there is a need for more information on the relative cost-effectiveness of interventions to help policymakers choose the ones that maximize student learning but remain within school budgets, which are often tight.

A WestEd study from Feng and colleagues analyzed the cost and impact of the ASSISTments program, an online, 1-to-1 homework support program for math which can be used in and out of the classroom to provide students with instant feedback and teachers with real-time progress reporting on their students. The original impact study (a randomized control trial of 5,912 7th grade students from 63 schools in 41 districts in North Carolina) showed a positive impact of ES=+0.10, which was sustained one year after the program ended.  

The current study builds on this evidence by estimating the cost required to produce these gains. Specifically, they estimated that it cost $46.23 per student for this 0.1 standard deviation increase in math achievement. This estimate (and cost-effectiveness ratio) is quite low compared to many other similar interventions and is more impressive given the rigorous research design that produced the estimated impacts (with randomization; large, diverse sample; and long-term impact estimate).

This study provides a guide for a quick application of cost-analysis methods, including a breakdown of each component’s contribution to the cost (Table 1). Additionally, the authors discuss the potential for alternate implementation approaches and how to integrate them into the cost-analysis.

Source (Open Access): Feng, M., Weiser, G., & Collins, K. (2024). Cost and cost effectiveness of ASSISTments online math support: Analysis from a randomized controlled study in middle school. WestEd. Report_Cost-and-Cost-Effectiveness-of-ASSISTments-Online-Math-Support_FINAL-ADA-v2… Read the rest