Category: Programme Evaluation

Categories
Kindergarten Programme Evaluation

The effect of robot programming versus unplugged programming on computational thinking and executive functions in preschool children: a randomized controlled trial

A recent study investigated whether robot programming offers added benefits over unplugged programming for developing preschoolers’ computational thinking (CT) and executive functions (EFs), given limited comparative evidence in early childhood contexts.

The authors conducted a randomized controlled trial with 198 children aged 5 to 6 from one public kindergarten in China. Children were randomly assigned to a robot programming group using the Matatalab kit (n = 66), an unplugged programming group using paper and pencil based activities (n = 66), or a business as usual control group engaging in conventional kindergarten activities (n = 66). The intervention lasted 12 weeks, with one 60 minute session per week. CT was assessed with TechCheck K (15 items; Cronbach’s alpha = 0.79 in this study), and EFs were measured with the Early Year Toolbox tasks assessing inhibition (Go/No-Go), working memory (Mr. Ant), and cognitive flexibility (Card Sorting). Measures were administered at baseline, week 6, and week 12, and effects were analysed using linear mixed effects models; implementation fidelity was reported as 97 percent adherence.

Results showed that both robot programming and unplugged programming groups outperformed the control group on CT over time, and the robot programming group showed stronger CT gains than the unplugged group by week 12. For EFs, the robot programming group outperformed both the unplugged programming and control groups over time on inhibition, working memory, and cognitive flexibility; within group analyses indicated significant improvements after 12 weeks only in the robot programming group for these EF outcomes. Most children in the robot programming group reported positive perceptions of programmable robots, including ease of use (79%), perceived usefulness (91%), technology anxiety (91%), satisfaction (94%), attitude (82%), and intention to continue use (85%).

The study suggests that while both modalities can support preschoolers’ CT, robot programming may produce more substantial and sustained benefits, particularly for EFs. The authors recommend further research with more diverse samples, longer follow up, mixed quantitative and qualitative evidence, and continued validation of EF measures and programming tools.

 

Source (Open Access): Zhang, X., Chen, Y., Hu, L., Hwang, G. J., & Tu, Y. F. (2025). Developing preschool children’s computational thinking and executive functions: unplugged vs. robot programming activities. International Journal of STEM Education, 12(1), 10.

https://doi.org/10.1186/s40594-024-00525-z… Read the rest

Categories
Effective Teaching Approach K-12 Education Programme Evaluation

Equity in Teacher Education Programs: Conceptions and Program-Level Efforts

Ko-Wong (2025) conducts a systematic review to examine how teacher education programs (TEPs) in the United States conceptualize, implement, and prioritize equity. Drawing on 58 empirical studies that met rigorous inclusion criteria, the review synthesizes program-level equity efforts across coursework, field placements, recruitment strategies, faculty development, and structural reforms. The analysis highlights substantial conceptual ambiguity: many programs invoke equity rhetorically while relying on race-evasive framings, thin interpretations of fairness, or narrow emphases on access and achievement. Few studies explicitly address Whiteness, racism, power, or meritocracy—core constructs in critical equity frameworks.

Using an adapted strong-equity lens, the review finds that most TEPs focus on surface-level or fragmented activities, such as required multicultural courses or isolated field experiences, which often lack coherence and long-term impact. Although coursework and community-based placements can support preservice teachers’ awareness of diverse learners, evidence suggests that these efforts frequently fall short of transforming candidates’ racial literacy or challenging institutional hierarchies. Only a limited subset of programs adopt more systemic approaches that integrate equity across curricula, partnerships, supervision, and program structures.

The findings underscore the persistent gap between equity rhetoric and equity enactment in teacher preparation. Ko-Wong argues that meaningful progress requires programs to name and confront systemic racism, destabilize dominant ideologies such as color-evasiveness and meritocracy, and reimagine TEPs as sites of structural change rather than individual skill development. The review concludes with recommendations for advancing strong equity through conceptual clarity, coherent program design, and deeper engagement with racial justice frameworks.

Source (Open Access): Ko-Wong, L. (2025). Equity in Teacher Education Programs: A Systematic Review of Conceptions and Program-Level Efforts. Review of Educational Research, 00346543251382579.

https://doi.org/10.3102/00346543251382579… Read the rest

Categories
Higher Education Programme Evaluation

Leap, learn, earn: exploring academic risk taking and learning success across gender and socioeconomic groups

A recent cross-sectional study by Hübner and Pfost (2024) investigates gender-related and SES-related disparities in academic risk-taking (ART) and their implications for learning success in undergraduate education. Drawing on a sample of 381 German university students, the study addresses four research questions: (1) Are there significant differences in ART levels based on students’ social group affiliation (gender, SES)? (2) Does ART predict students’ learning success? (3) Are there significant indirect effects of social group affiliation on learning success via ART (mediation)? (4) Are there significant differences in the strength of the relationship between ART and learning success depending on students’ social group affiliation (moderation)?

The authors employed structural equation modeling to address these questions. Specifically, ART was assessed across two dimensions—seminar group and peer group—while learning success was measured through students’ subjective evaluations of their own learning achievement in the current seminar. Gender was coded as female, male, or diverse, with the latter category excluded due to its small sample size. SES was dichotomized into high and low groups. Higher education entry qualification (HEEQ) was included as a general control variable, based on students’ self-reported prior academic performance (ranging from 1 = very good to 6 = insufficient).

The findings reveal significant gender differences in ART on both the seminar group dimension (Mmale = 3.53, Mfemale = 3.23, F = 11.83, p = 0.001, d = 0.40) and the peer group dimension (Mmale = 3.27, Mfemale = 3.49, F = 5.58, p = 0.018, d = 0.28). In contrast, no significant differences emerged between high-SES students and low-SES students in either the seminar group dimension (Mlow-SES = 3.26, Mhigh-SES = 3.33, F = 0.74, p = 0.390, d = 0.09) or the peer group dimension (Mlow-SES = 3.39, Mhigh-SES = 3.45, F = 0.51, p = 0.476, d = 0.07). Both the seminar group (β = 0.23, p = 0.004) and peer group (β = 0.21, p = 0.009) dimensions of ART significantly predicted learning success. Regarding mediation effects, the indirect path from gender through the seminar group dimension of ART to learning success was significant (β = -0.04, p = 0.015), whereas the peer group dimension did not function as a mediator. Furthermore, gender (β = 0.10, p = 0.004) and SES (β = 0.10, p = 0.018) significantly moderated the relationship between the peer group dimension of ART and learning success, but no moderation effects were observed for the seminar group dimension.

This study is among the first to confirm gender differences in ART within higher education and to demonstrate the beneficial effects of ART on learning success using inferential statistical methods. By addressing the four research questions, it contributes to a more nuanced understanding of ART and its role in the reproduction of educational inequalities.

 

Source (Open Access): Hübner, V., & Pfost, M. (2024). Leap, learn, earn: exploring academic risk taking and learning success across gender and socioeconomic groups. Higher Education, 1-19.

 https://doi.org/10.1007/s10734-024-01307-w… Read the rest

Categories
Programme Evaluation Secondary School Education

Long-term effects of grade retention for older students

Research suggests that early grade retention is less harmful than in later grades, when staying with one’s peers s especially crucial. Matthew Larsen and Jon Valant examined a Louisiana policy that assigned students with low test scores to three different placements: full retention in grade 8, promotion to grade 9, or a hybrid “grade 8.5” option where students entered high school but received 8th-grade-level instruction. This hybrid approach aimed to reduce the stigma of retention by keeping students with their peers while offering academic remediation.

Using regression discontinuity designs, the authors found that the higher a student was placed, the more likely they were to graduate from high school and enroll in college across all comparisons: grade 9 vs 8, grade 9 vs grade 8.5, and grade 8.5 vs 8.  Students who were near the grade retention scores for 8th grade were less likely to enroll in college. The authors concluded that retaining older students, even with remediation, may undermine long-term educational outcomes.

Source: Larsen, M. F., & Valant, J. (2024). The long-term effects of grade retention: Evidence on persistence through high school and college. Journal of Research on Educational Effectiveness, 17(4), 615-646. https://doi.org/10.1080/19345747.2023.2240323… Read the rest

Categories
Educational Administration and Leadership Primary School Education Programme Evaluation

The impact of tutoring programs at scale

Kraft and colleagues conducted a meta-analysis of 265 randomized controlled trials evaluating the effects of 340 tutoring programs to understand what impacts should be expected from tutoring programs when implemented at scale in the U.S., using standardized tests as outcome measures. Most of the included studies were conducted in elementary school and in reading.

The results showed that as the number of students served by the tutoring programs increased, the effects tended to decline. The average effect size was +0.44 for programs with fewer than 99 students and +0.30 for those with 100–399 students. For large-scale tutoring studies, the effect size was +0.21 with 400–999 students and +0.16 with more than 1,000 students. The authors noted that these effects remain substantial. However, similar effects to those observed in the full set of meta-analytic studies should not be expected when tutoring is implemented at scale. The wide variability in effect sizes also suggests that individual programs differ considerably in effectiveness.

The authors tested hypotheses to explain this pattern. One possible reason is that it becomes more difficult to maintain high-quality implementation as the number of students increases. Another is that, in large-scale evaluations, program features are often adjusted to make tutoring more feasible—for example, by assigning each tutor to larger groups of students.

The authors concluded that the effects of tutoring observed in their study remain meaningful and relevant for both practice and policy. But it is also important to maintain realistic expectations about the impact of tutoring when it is implemented broadly in real-world school settings.

 

Source (Open Access): Kraft, Matthew A., Beth E. Schueler, and Grace Falken. (2024). What Impacts Should We Expect from Tutoring at Scale? Exploring Meta-Analytic Generalizability. (EdWorkingPaper: 24-1031). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/zygj-m525… Read the rest

Categories
Educational Administration and Leadership Kindergarten Programme Evaluation

Could a short professional development program enhance children’s science learning outcomes?

A recent experimental study by Guan and Hu in southeast China investigated two approaches to improving early childhood pre-service teachers’ science teaching abilities and children’s learning outcomes:

  • The Making the Most of Classroom Interactions (MMCI) course alone
  • An Integrated Intervention Program (IIP) that combined MMCI with peer coaching

The MMCI course, based on the Classroom Assessment Scoring System (CLASS) framework, consisted of four 2-hour sessions focused on effective instructional support strategies, utilizing video analysis and sharing of strategy implementations. The peer coaching is a collaborative approach which involved teachers in small groups reviewing each other’s teaching videos and providing structured feedback.

The researchers randomly assigned 87 pre-service teachers, who received standard training, to three conditions: MMCI alone (n=31), IIP (MMCI plus peer coaching, n=30), and a control group (n=26). The intervention lasted four weeks, and teachers’ beliefs, knowledge, skills, and classroom practices were measured through self-reported questionnaire, video-based test, and teaching video assessment. Researchers assessed learning outcomes for 305 children (mean age = 5.69) in these teachers’ classrooms.

The findings revealed that both MMCI and IIP significantly improved teachers’ science teaching beliefs, knowledge, instructional practices, and skills. Both approaches enhanced children’s science problem-solving skills and belief about the ease of learning science, though no significant improvements were found on science competence, science liking, or concept understanding.

Notably, the IIP group demonstrated greater gain in teachers’ teaching knowledge and children’s problem-solving abilities compared to MMCI-only group. The study highlights the importance of integrating structured training like MMCI with collaborative peer coaching to maximize pre-service teacher development and children’s learning.

 

Source: Guan, L., & and Hu, B. Y. (n.d.). Effects of the MMCI instructional support intervention and peer coaching on ECE pre-service teachers’ science teaching and children’s learning. Journal of Research on Educational Effectiveness, 0(0), 1–29. https://doi.org/10.1080/19345747.2025.2493653… Read the rest