Tag: Meta-analysis

Categories
K-12 Education Language Development Maths and Science Learning

Are mathematics and writing skills related?

A substantial body of research has established links between reading and mathematics skills, as well as between reading and writing. Moreover, previous studies suggest that these skills share abilities such as executive function and higher-order cognition. It is therefore reasonable to hypothesize a connection between mathematics and writing skills.

A meta-analysis by Kim and colleagues investigated the correlation between mathematics and writing skills. The authors categorized both writing and mathematics skills into lower-order and higher-order subskills. In mathematics, skills involving information retrieval and understanding of magnitude (e.g., arithmetic, calculation fluency) are considered lower-order or foundational skills, whereas skills involving reasoning and comprehension (e.g., word-problem solving, data interpretation) are higher-order skills. In writing, transcription skills (spelling and handwriting/keyboarding) are lower-order, while written composition is a higher-order skill.

The meta-analysis included 211 studies with 564 effect sizes, primarily from English-speaking participants. Utilizing robust variance estimation, the results showed that the overall correlation between mathematics and writing skills was moderate (r = .48), with grade level significantly moderating this correlation. The effect was found strongest among lower primary students (K-Grade 2: r = 0.52), decreased through upper elementary (Grade 3-6: r=0.42), to college students and adults levels (r = 0.30).

Given the potential overlap between grade level and skill level, the analysis was further disaggregated by grade level. In primary grades, the lower-order writing and lower-order mathematics skills were moderately correlated (r=0.59), while the correlation between higher-order writing and higher-order mathematics was 0.48. Among university learners, the lower-order link was 0.36, and the higher-order link was 0.25.

The findings support a substantial link between mathematics and writing skills. However, the correlations decreased as grade level increased, and the higher-order link was comparatively weaker. The authors speculated a twofold explanation: a plateau of foundational skills and the possibility that different cognitive skills might be involved in higher-order tasks, warranting further exploration.

 

Source (Open Access): Kim, Y.-S. G., Yang, D., & Hwang, J. (2024). Are mathematics and writing skills related? Evidence from meta-analysis. Educational Psychology Review, 36(4), 125. https://doi.org/10.1007/s10648-024-09960-4… Read the rest

Categories
Primary School Education Secondary School Education Social and Motivational Outcomes

Are counselor-led SEL interventions effective? Insights from a recent meta-analysis

Social and emotional learning (SEL) interventions help students, educators, and school staff manage emotions, build relationships, and make thoughtful decisions. These programs are linked to improved academic performance and overall well-being. School counselors are crucial in implementing SEL programs but often face challenges due to limited resources, such as limited time and training. As students face increasing mental and social challenges, understanding the impact and best practices for these interventions is essential.

A recent meta-analysis reviewed the effectiveness of school counselor-led SEL interventions, including 28 studies published over the past 20 years. Using a random effects model, the analysis found an average statistically significant positive effect (ES = +0.31). Subgroup analyses and meta-regression revealed that while no moderators showed statistically significant differences, several factors were associated with larger effect sizes. Notably, larger effect sizes were observed in randomized controlled trials (RCTs) (ES =+0.31) and cluster RCTs (ES = +0.42) in samples with over 50% racial/ethnic minority students (ES = +0.40 versus ES = +0.21 in samples with less diversity). Effects were larger for low SES groups (ES = +0.35 compared to +0.21 for higher SES group), and for middle (ES = +0.45) and high school students (ES = +0.43) compared to elementary students (ES = +0.08).

The interventions were more effective for academic development (ES = +0.41) than for social-emotional (ES = +0.19) or career development (ES = +0.14). Small group interventions showed the highest effect (ES = +0.44), followed by classroom lessons (ES = +0.30). Interventions grounded in theory had larger effects (ES = +0.41 for multiple theories, ES = +0.32 for a single theory) compared to those without theoretical grounding (ES = +0.17). Applying theory to measure changes in relevant constructs led to larger effects (ES = +0.34) compared to those that did not (ES = +0.26), and discussing results in relation to theory also showed higher effect sizes (ES = +0.40 vs. +0.27).

Despite the limitation of including only peer-reviewed published papers, which may lead to an overrepresentation of positive findings, this meta-analysis suggests that school counselors contribute significantly to SEL-related outcomes, especially for academic achievement. Additionally, the integration of theory in SEL programming seems to be associated with more impactful intervention results.

 

Source: Lemberger‐Truelove, M. E., Li, D., Kim, H., Wills, L., Thompson, K., & Lee, Y. (2024). Meta‐analysis of social and emotional learning interventions delivered by school counselors. Journal of Counseling & Development, jcad.12537. https://doi.org/10.1002/jcad.12537… Read the rest

Categories
Achievement Educational Administration and Leadership K-12 Education Social and Motivational Outcomes

Effectiveness of flipped classrooms on student outcomes

Flipped classroom refers to an instructional approach where students first learn new material through videos outside of class, and then use class time for reviewing the content, hands-on activities, and deeper learning.

A meta-analysis conducted by Li and colleagues examined the effects of flipped classrooms on K–12 student outcomes. The authors included randomized and quasi-experimental studies with more than ten students in which flipped classrooms were compared to regular teacher practice. After excluding the studies that did not meet the criteria, 129 studies were included in the review.

The average effect sizes after excluding outliers were +0.53 for student performance, +0.56 for cognitive outcomes, and +0.46 for affective outcomes. Due to the high variability in effects between studies, the authors examined which characteristics are factors (e.g., duration, teacher training) that moderate the effects of flipped classrooms. Region was found to be a significant moderator, with a larger effect in low- and middle-income countries compared to high-income countries. Considering methodological characteristics, the authors found that it is more effective when the same instructor teaches consistently than when instructors change during an intervention. In addition, outcomes that include a wide range of tests show smaller effect sizes than those that use more specific measures, such as a unit test.

Overall, the characteristics related to educational context and research methodology explained a substantial portion of the variability in the overall effects.

 

Source: Li, S., Fu, W., Liu, X., & Hwang, G.-J. (2024). Effectiveness of flipped classrooms for k–12 students: Evidence from a three-level meta-analysis. Review of Educational Research, 00346543241261732. https://doi.org/10.3102/00346543241261732… Read the rest

Categories
Educational Administration and Leadership K-12 Education

The impact of COVID on learning

As federal relief funds for education are about to expire just as schools are opening for the new school year, the effects of COVID-19 on education and student learning remain significant. NWEA MAP assessments have shown that students in the United States have still not returned to pre-COVID learning levels. A recent study by Storey & Zhang investigated the extent of learning losses caused by the pandemic and the impact of school closures and disruptions on various student groups.

Using meta-analytic research methods, the authors identified 30 studies from the U.S. and similar countries that used standardized assessments to measure student achievement before, during, and after the pandemic (with a 2022 cutoff). The analysis illustrated that students lost 0.21 (p= .006) standard deviations of learning during the pandemic school closures, with losses of 0.43 standard deviations in reading and 0.41 standard deviations in mathematics. 

These results highlight the significant and statistically meaningful impact of COVID-19 on education, providing a clearer understanding of the extent of the disruptions. The findings underscore the need for ongoing federal and state support to help educators and school systems address and recover from these losses.

 

Source: Storey, N., & Zhang, Q. (2024). A meta-analysis of the impact of COVID-19 on student achievement. Educational Research Review, 44, 100624. https://doi.org/10.1016/j.edurev.2024.100624… Read the rest

Categories
Achievement Educational Administration and Leadership K-12 Education

Can metacognition predict academic achievement? A meta-analysis of longitudinal studies

Metacognition refers to students’ self-regulation abilities in their learning process, involving planning, monitoring, and assessing one’s understanding and performance. For instance, the degree to which students monitor their progress during preparing for exams and recognize necessary adjustment for academic success. Previous studies indicated early metacognitive predicts subsequent academic achievement. Though meta-analyses examined the relationships between metacognition and academic outcomes, relying primarily on cross-sectional studies would risk bias from common method variance. To address this, He and colleagues conducted a three-level meta-analysis utilizing only longitudinal studies.

The meta-analysis included 28 studies, yielding 66 effect sizes, published between 1990 and 2023. These studies featured time-lag between measures of metacognition and academic outcomes ranging from 3 to 24 months, with an average of 10.3 months. Results revealed a small but significant positive correlation between initial metacognition and subsequent academic achievement (r = +0.22). Moderator analyses found on-line measures (r = +0.32), which utilized behavioral measure (e.g., on-going task performance), were stronger predictors than off-line measures (r=+0.17), which mainly used self-report questionnaire, after controlling other moderators. Effects also varied by metacognition components with largest effect for metacognitive skill (r = +0.34), followed by metacognitive awareness (r=+0.22), metacognitive strategy study (r = +0.21), and metacognitive knowledge (r = +0.17).

Additionally, examining the reversal link, results found a small positive correlation between initial academic achievement and subsequent metacognition (r=+0.12). Furthermore, autoregressive cross-lagged meta-analytic structural equation modelling revealed significant small bidirectional associations: students’ metacognition predicted their subsequent academic performance (beta = +0.038), and their academic outcomes also predicted future metacognition (beta = +0.079).

While longitudinal data shed light on dynamic relationships, the authors caution that causal inferences cannot be definitively drawn from fitting hypothesized model. Overall, the meta-analysis provides evidence that metacognition and achievement reciprocally and weakly predict each other over time.

 

Source: He, G., Chen, S., Lin, H., & Su, A. (2024). The association between initial metacognition and subsequent academic achievement: A meta-analysis of longitudinal studies. Educational Psychology Review, 36(3), 81. https://doi.org/10.1007/s10648-024-09922-w… Read the rest

Categories
Primary School Education Secondary School Education Social and Motivational Outcomes

The fadeout effect: Unpacking the durability of educational interventions

Although educational interventions play a pivotal role in fostering a broad range of student skills, their sustained impact often wanes as time progresses, prompting a deeper exploration into their long-term efficacy. Hart and colleagues conducted a comprehensive meta-analysis, examining the long-term effectiveness of educational interventions aimed at improving cognitive and social-emotional skills in children and adolescents. The study drew on 86 randomized controlled trials (RCTs), incorporating data from 56,662 participants, to determine whether the gains in social-emotional skills were more persistent than those in cognitive skills. The research included studies from eight previous meta-analyses and aimed to overcome the methodological limitations of earlier works by focusing solely on RCTs and outcomes consistently measured across different time points.

The analysis revealed that both cognitive and social-emotional skills demonstrated a significant fadeout in their effects over time. Initial post-test improvements did not persist strongly at follow-up assessments, challenging the expectation that social-emotional skills would endure more robustly than cognitive skills. The study found similar rates of conditional persistence for both types of skills in the short term (6-12 months). However, cognitive skills exhibited slightly greater persistence at longer follow-up intervals (1-2 years).

This meta-analysis highlights the critical need for educational programs that not only focus on short-term gains but also address strategies to maintain these improvements over time. The findings suggest that without sustained intervention, both cognitive and social-emotional skills are subject to significant declines, which calls into question the long-term efficacy of current educational interventions in altering developmental trajectories. The results also emphasize the importance of designing future educational programs with a focus on long-lasting impact, a challenge that remains central to educational research and practice.

 

Source (Open Access): Hart, Emma R., Drew H. Bailey, Sha Luo, Pritha Sengupta, and Tyler W. Watts. (2024). Fadeout and persistence of intervention impacts on social-emotional and cognitive skills in children and adolescents: A meta-analytic review of randomized controlled trials. (EdWorkingPaper: 23-782). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/7J8S-DY98… Read the rest