Category: Programme Evaluation

Categories
Higher Education Programme Evaluation

Leap, learn, earn: exploring academic risk taking and learning success across gender and socioeconomic groups

A recent cross-sectional study by Hübner and Pfost (2024) investigates gender-related and SES-related disparities in academic risk-taking (ART) and their implications for learning success in undergraduate education. Drawing on a sample of 381 German university students, the study addresses four research questions: (1) Are there significant differences in ART levels based on students’ social group affiliation (gender, SES)? (2) Does ART predict students’ learning success? (3) Are there significant indirect effects of social group affiliation on learning success via ART (mediation)? (4) Are there significant differences in the strength of the relationship between ART and learning success depending on students’ social group affiliation (moderation)?

The authors employed structural equation modeling to address these questions. Specifically, ART was assessed across two dimensions—seminar group and peer group—while learning success was measured through students’ subjective evaluations of their own learning achievement in the current seminar. Gender was coded as female, male, or diverse, with the latter category excluded due to its small sample size. SES was dichotomized into high and low groups. Higher education entry qualification (HEEQ) was included as a general control variable, based on students’ self-reported prior academic performance (ranging from 1 = very good to 6 = insufficient).

The findings reveal significant gender differences in ART on both the seminar group dimension (Mmale = 3.53, Mfemale = 3.23, F = 11.83, p = 0.001, d = 0.40) and the peer group dimension (Mmale = 3.27, Mfemale = 3.49, F = 5.58, p = 0.018, d = 0.28). In contrast, no significant differences emerged between high-SES students and low-SES students in either the seminar group dimension (Mlow-SES = 3.26, Mhigh-SES = 3.33, F = 0.74, p = 0.390, d = 0.09) or the peer group dimension (Mlow-SES = 3.39, Mhigh-SES = 3.45, F = 0.51, p = 0.476, d = 0.07). Both the seminar group (β = 0.23, p = 0.004) and peer group (β = 0.21, p = 0.009) dimensions of ART significantly predicted learning success. Regarding mediation effects, the indirect path from gender through the seminar group dimension of ART to learning success was significant (β = -0.04, p = 0.015), whereas the peer group dimension did not function as a mediator. Furthermore, gender (β = 0.10, p = 0.004) and SES (β = 0.10, p = 0.018) significantly moderated the relationship between the peer group dimension of ART and learning success, but no moderation effects were observed for the seminar group dimension.

This study is among the first to confirm gender differences in ART within higher education and to demonstrate the beneficial effects of ART on learning success using inferential statistical methods. By addressing the four research questions, it contributes to a more nuanced understanding of ART and its role in the reproduction of educational inequalities.

 

Source (Open Access): Hübner, V., & Pfost, M. (2024). Leap, learn, earn: exploring academic risk taking and learning success across gender and socioeconomic groups. Higher Education, 1-19.

 https://doi.org/10.1007/s10734-024-01307-w… Read the rest

Categories
Programme Evaluation Secondary School Education

Long-term effects of grade retention for older students

Research suggests that early grade retention is less harmful than in later grades, when staying with one’s peers s especially crucial. Matthew Larsen and Jon Valant examined a Louisiana policy that assigned students with low test scores to three different placements: full retention in grade 8, promotion to grade 9, or a hybrid “grade 8.5” option where students entered high school but received 8th-grade-level instruction. This hybrid approach aimed to reduce the stigma of retention by keeping students with their peers while offering academic remediation.

Using regression discontinuity designs, the authors found that the higher a student was placed, the more likely they were to graduate from high school and enroll in college across all comparisons: grade 9 vs 8, grade 9 vs grade 8.5, and grade 8.5 vs 8.  Students who were near the grade retention scores for 8th grade were less likely to enroll in college. The authors concluded that retaining older students, even with remediation, may undermine long-term educational outcomes.

Source: Larsen, M. F., & Valant, J. (2024). The long-term effects of grade retention: Evidence on persistence through high school and college. Journal of Research on Educational Effectiveness, 17(4), 615-646. https://doi.org/10.1080/19345747.2023.2240323… Read the rest

Categories
Educational Administration and Leadership Primary School Education Programme Evaluation

The impact of tutoring programs at scale

Kraft and colleagues conducted a meta-analysis of 265 randomized controlled trials evaluating the effects of 340 tutoring programs to understand what impacts should be expected from tutoring programs when implemented at scale in the U.S., using standardized tests as outcome measures. Most of the included studies were conducted in elementary school and in reading.

The results showed that as the number of students served by the tutoring programs increased, the effects tended to decline. The average effect size was +0.44 for programs with fewer than 99 students and +0.30 for those with 100–399 students. For large-scale tutoring studies, the effect size was +0.21 with 400–999 students and +0.16 with more than 1,000 students. The authors noted that these effects remain substantial. However, similar effects to those observed in the full set of meta-analytic studies should not be expected when tutoring is implemented at scale. The wide variability in effect sizes also suggests that individual programs differ considerably in effectiveness.

The authors tested hypotheses to explain this pattern. One possible reason is that it becomes more difficult to maintain high-quality implementation as the number of students increases. Another is that, in large-scale evaluations, program features are often adjusted to make tutoring more feasible—for example, by assigning each tutor to larger groups of students.

The authors concluded that the effects of tutoring observed in their study remain meaningful and relevant for both practice and policy. But it is also important to maintain realistic expectations about the impact of tutoring when it is implemented broadly in real-world school settings.

 

Source (Open Access): Kraft, Matthew A., Beth E. Schueler, and Grace Falken. (2024). What Impacts Should We Expect from Tutoring at Scale? Exploring Meta-Analytic Generalizability. (EdWorkingPaper: 24-1031). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/zygj-m525… Read the rest

Categories
Educational Administration and Leadership Kindergarten Programme Evaluation

Could a short professional development program enhance children’s science learning outcomes?

A recent experimental study by Guan and Hu in southeast China investigated two approaches to improving early childhood pre-service teachers’ science teaching abilities and children’s learning outcomes:

  • The Making the Most of Classroom Interactions (MMCI) course alone
  • An Integrated Intervention Program (IIP) that combined MMCI with peer coaching

The MMCI course, based on the Classroom Assessment Scoring System (CLASS) framework, consisted of four 2-hour sessions focused on effective instructional support strategies, utilizing video analysis and sharing of strategy implementations. The peer coaching is a collaborative approach which involved teachers in small groups reviewing each other’s teaching videos and providing structured feedback.

The researchers randomly assigned 87 pre-service teachers, who received standard training, to three conditions: MMCI alone (n=31), IIP (MMCI plus peer coaching, n=30), and a control group (n=26). The intervention lasted four weeks, and teachers’ beliefs, knowledge, skills, and classroom practices were measured through self-reported questionnaire, video-based test, and teaching video assessment. Researchers assessed learning outcomes for 305 children (mean age = 5.69) in these teachers’ classrooms.

The findings revealed that both MMCI and IIP significantly improved teachers’ science teaching beliefs, knowledge, instructional practices, and skills. Both approaches enhanced children’s science problem-solving skills and belief about the ease of learning science, though no significant improvements were found on science competence, science liking, or concept understanding.

Notably, the IIP group demonstrated greater gain in teachers’ teaching knowledge and children’s problem-solving abilities compared to MMCI-only group. The study highlights the importance of integrating structured training like MMCI with collaborative peer coaching to maximize pre-service teacher development and children’s learning.

 

Source: Guan, L., & and Hu, B. Y. (n.d.). Effects of the MMCI instructional support intervention and peer coaching on ECE pre-service teachers’ science teaching and children’s learning. Journal of Research on Educational Effectiveness, 0(0), 1–29. https://doi.org/10.1080/19345747.2025.2493653… Read the rest

Categories
Educational Administration and Leadership K-12 Education Programme Evaluation

Cross-age tutoring: Are adult tutors better than student tutors?

A meta-analysis published in Educational Psychology Review by Chang and colleagues provides comprehensive evidence on the effectiveness of cross-age tutoring in educational settings. Cross-age tutoring involves pairing an older tutor with a younger tutee at different developmental stages, with the expectation that both participants benefit. This meta-analysis analyzed 31 experimental studies (RCT or quasi experimental design) involving147 effect sizes of academic outcomes. Of these, 28 studies (117 effect sizes) involved tutees, while 14 studies (30 effect sizes) examined the impact on tutors. The tutors ranged from older students in grades 5 through 12 to adult tutors, including university students and community volunteers.

The researchers found that cross-age tutoring generated positive academic outcomes for both tutors and tutees, with an overall effect size of 0.34. Specifically, tutees showed significant academic improvement (ES=0.33), while tutors also demonstrated considerable learning gains (ES=0.39). Importantly, the analysis revealed that student tutors (ES = 0.37) were as effective as adult tutors (ES = 0.25) on tutees, and the intervention proved beneficial for both typically developing students and those with learning difficulties. The positive effects were consistent across both reading and mathematics subjects.

The findings have some implications for educational practice and policy decisions. For instance, the schools can effectively utilize older students as tutors, which offers both economic advantages and feasibility. However, the researchers note several areas requiring further investigation, including cost-effectiveness analysis comparing peer tutoring with adult tutoring, optimal tutoring duration and frequency.

 

Source (Open Access): Chang, A., Mauer, E., Wanzek, J., Kim, S., Scammacca, N., & Swanson, E. (2025). Examining the academic effects of cross-age tutoring: A meta-analysis. Educational Psychology Review, 37(1), 19. https://doi.org/10.1007/s10648-025-09997-z… Read the rest

Categories
Educational Administration and Leadership Kindergarten Programme Evaluation

Can a video-based parent-child program improve preschooler’s executive function?

Growing evidence indicates that executive function (EF) plays a vital role in enhancing children’s learning, and EFs are responsive to intervention. A recent study by Lau and colleagues investigated the effects of the Parent-Child Brain Camp (The Camp), a video-based intervention program designed to enhance EF skills in children aged 5 to 6. The Camp incorporates play-based and age-appropriate interactive activities to three targeted core EF components: working memory, inhibitory control, and cognitive flexibility. Owing to the Covid-19 pandemic lockdown, the program was conducted online in the home environment in spring 2022. It consisted of 12 video Zoom sessions, each lasting about 20 minutes, with parental involvement to support children’s motivation and engagement.

Participants were Chinese middle-class families from Hong Kong, recruited from 6 kindergartens and social media. Eligible families were randomly assigned to either the treatment group or control group. The treatment group received 3 video interventions weekly for 4 weeks. The analytic sample comprised 79 children in the treatment group and 74 in the control group, with a mean age of 68.9 months. Pre-test and post-test scores were collected via Zoom for four EF assessment tasks. Results revealed that the treatment group showed significantly higher improvement in composite EF scores. Specifically, they had significantly higher increment on the Mr. Cucumber task compared to control group.

The study demonstrated a brief period of video-based intervention program could enhance young children’s EFs. While parents’ engagement may be an important factor in these improvement, its specific effect could not be isolated. Moreover, the long-term effects require further investigation, as does the reliability and validity of online EF assessments.

 

Source (Open Access): Lau, E. Y. H., Wu, X., Siu, C. T. S., Williams, K. E., & Bautista, A. (2025). Examining the effectiveness of a video-based parent–child program on executive functions for children 5 to 6 years old: A randomized controlled trial. Child Development, 96(2), 781–796. https://doi.org/10.1111/cdev.14208… Read the rest