Tag: Meta-analysis

Categories
Language Development Primary School Education Programme Evaluation Secondary School Education

Optimizing reading comprehension intervention : A Bayesian network meta-analysis

Peng and colleagues conducted a Bayesian network meta-analysis (BNMA) to evaluate the effectiveness of various reading comprehension strategies for students with reading difficulties (RD) in grades 3 to 12. They included 52 experimental/quasi-experimental studies which examined strategies including main idea (or summarizing), inference, text structure, retell, prediction, self-monitoring, and graphic organizers, along with their combinations. The researchers explored potential moderators like background knowledge instruction, age, text type, RD status, assessment type, intervention dosage, and study quality.

Unlike traditional meta-analysis, network meta-analysis can compare multiple interventions simultaneously and rank their effectiveness, even when some interventions haven’t been directly compared in individual studies, based on both direct and indirect evidence. The study found that the most effective strategy combination was the main idea-text structure-retell (MTR) combination (ES =+1.72, SUCRA = 0.89). Other effective strategies included the main idea-text structure-self-monitoring-graphic organizers (MTSG) combination (ES =+1.13, SUCRA = 0.76) and the main idea strategy alone (ES =+1.07, SUCRA=0.77). Background knowledge (vocabulary / content knowledge) was the only significant moderator that enhanced the effectiveness of these strategies. Without it, no strategies or combinations showed significant efficacy. The number of strategies in a combination did not influence the reading comprehension outcome, suggesting that teaching too many strategies could increase cognitive load, making it difficult for struggling readers to apply them effectively.

These insights can help educators design more effective reading interventions by focusing on optimal strategy combinations and incorporating background knowledge instruction to enhance comprehension for struggling readers.

 

Note: SUCRA = the surface under the cumulative ranking curve scores, which indicates how likely an intervention strategy would be evaluated as the most efficacious (the worst 0% to the best 100%).

 

Source (Open Access): Peng, P., Wang, W., Filderman, M. J., Zhang, W., & Lin, L. (2024). The Active Ingredient in Reading Comprehension Strategy Intervention for Struggling Readers: A Bayesian Network Meta-analysis. Review of Educational Research, 94(2), 228–267. https://doi.org/10.3102/00346543231171345… Read the rest

Categories
Achievement K-12 Education

Effects of educational technology on the performance of disadvantaged students

A recent meta-analysis examined the effects of technology-based educational interventions on the academic performance of disadvantaged K–12 students. The review included studies conducted in less developed countries or with at least 50% of students from disadvantaged backgrounds, defined by socio-economic status, minority status, or areas of residence. Studies needed to use technology in class, while courses delivered online were excluded.

The review included 72 studies with 740 effect sizes conducted in 24 different countries, although most of the studies took place in the US and in India. Ed-tech interventions showed an average effect on academic performance of +0.20 across all included studies with average effect sizes of +0.23 in more developed countries and +0.18 in less developed countries. Regarding factors in the interventions and school contexts that made them effective, slightly larger effects were found in math and science outcomes compared to humanities. The authors found that Computer-Assisted Learning (CAL; ES = +0.12) and behavioral interventions (ES = +0.09) had larger effects compared to interventions that just provided the access to technology (e.g., internet, tablets).

Behavioral approaches that involved sending messages to parents to keep them informed and engaged were particularly effective, especially in communities where education is undervalued and families have limited resources. This highlights the importance of family and community support in helping disadvantaged students succeed academically.

 

Source (Open Access): Di Pietro, G., & Castaño Muñoz, J. (2025). A meta-analysis on the effect of technology on the achievement of less advantaged students. Computers & Education, 226, 105197. https://doi.org/10.1016/j.compedu.2024.105197… Read the rest

Categories
Educational Administration and Leadership Language Development Primary School Education Programme Evaluation

A meta-analysis of effective Spanish reading interventions

A meta-analysis conducted by our Best Evidence in Brief (BEiB) colleagues José Luis Arco-Tirado, Amanda Neitzel, and the late Bob Slavin, examined the effects of Spanish reading interventions for students in grades K–6. The review, published in Review of Educational Research, included studies focused on class- or school-based interventions aimed at improving reading skills among students whose first language was Spanish. The analysis included only high-quality studies that met rigorous criteria, such as the use of experimental or quasi-experimental designs and quantitative standardized measures of reading achievement.

The final best-evidence synthesis and meta-analysis was based on 11 studies with 51 effect sizes. Overall, Spanish reading interventions showed a significant average effect size of +0.49 on reading performance. While results varied considerably, the researchers explored grade level and outcome type as potential moderators of the effect. However, due to the small number of studies, no significant differences were found between grades K–2 and 3–6 or across the five categories of outcome type.

The authors concluded, on one hand, that some effective interventions for improving Spanish reading comprehension are available to help readers in K–6 education. On the other hand, more rigorous evaluation studies have to be implemented to expand the pool of proven reading practices available for teachers and educators in Spanish-speaking countries.

 

Source (Open Access): Arco-Tirado, J. L., Fernández-Martín, F. D., Hervás-Torres, M., Jiménez-Fernández, G., Calet, N., Defior, S., Neitzel, A. J., & Slavin, R. E. (2024). A Best-Evidence Synthesis and Meta-analysis on Effective Reading Programs in Spanish. Review of Educational Research, 00346543241297668. https://doi.org/10.3102/00346543241297668… Read the rest

Categories
Maths and Science Learning Primary School Education

Inhibitory Control and Math Skills: What’s the Connection?

A recent three-level meta-analysis by Zhu and colleagues synthesized 241 effect sizes from 86 samples of primary school students (mean age = 8.67 years) from studies conducted between 2001 to 2023 to investigate how two types of inhibitory control relate to various math domains (e.g., numerical, arithmetic, logical reasoning, general). The researchers distinguished between two main types of inhibitory control: (1) Interference Inhibition involves filtering out distracting information, like resisting the bias that a larger denominator means a larger fraction; and (2) response Inhibition focuses on controlling impulsive actions, like carefully considering a problem before blurting out an answer.

After removing two outlier effect sizes, the meta-analysis revealed a small positive correlation (r=0.19) between inhibitory control and overall math ability indicating that better inhibitory control is correlated with better math skills. The relationship was significantly stronger for interference inhibition (r=0.21) than response inhibition (r=0.14). Subgroup analyses revealed that the correlations between interference inhibition and various math domains (ranging from r=0.18 to r=0.27) were generally stronger than those for response inhibition across these domains (ranging from r=-0.01 to r=0.19).

The relationship between inhibitory control and mathematical ability was not moderated by domains of mathematical ability and inhibitory control task. However, pairwise comparison indicated that the link between arithmetical ability and interference inhibition (r = 0.23) was significantly stronger than the correlation between arithmetical ability and response inhibition (r = 0.12). Moreover, inhibit control measured by the Random generation task (r=0.39) was more strongly associated with math ability than when measured by the Go/No-go task (r=0.17) and other tasks (r=0.16). No significant moderation effects were found among demographic characteristics such as age, developmental status, and sample region.

This research highlights the stable influence of inhibitory control on math ability across various mathematical domains, measurement tasks, and demographic characteristics. The practical implications of these findings are discussed within the paper.

 

Source: Zhu, X., Tang, Y., Lu, J., Song, M., Yang, C., & Zhao, X. (2024). Inhibitory control and mathematical ability in elementary school children: A preregistered meta-analysis. Educational Psychology Review, 37(1), 1. https://doi.org/10.1007/s10648-024-09976-w… Read the rest

Categories
Higher Education K-12 Education Language Development

Will individuals who read well also write well?

Kim and colleagues conducted a meta-analysis to explore the relationship between reading and writing skills, particularly focus on how this relation varies by linguistic grain size, measurement of reading comprehension and written composition, and grade levels. Using robust variance estimation to address the nested structure of effect sizes, the analysis included 395 studies with 646 unique samples, totalling 2,265 effect sizes, and found a strong average correlation of 0.72 between reading and writing.

Across various linguistic grain sizes (units or chunks of language), the results showed that word reading and spelling had the strongest relation (r = 0.82), while reading comprehension and written composition had a moderate relation (r = 0.44). Other moderate correlations included spelling and text reading fluency (r = 0.59), word reading and written composition (r = 0.42), and text reading fluency and written composition (r = 0.33).

Further investigation highlighted relations between different of reading comprehension tasks and various composition dimensions. Among various measures of reading comprehension, writing quality had stronger correlations with oral-retell (r = 0.55), open-ended questions (0.49), and multiple choice (r=0.48) than the cloze task (0.37). Regarding dimensions of written composition, reading comprehension correlated more strongly with writing conventions (0.55), writing quality (0.46), and writing syntax (0.41) than with writing productivity (r0.23).

Developmentally, the word reading and spelling relation was significantly stronger for primary grade students (r = 0.82) than for university students and adults (0.69). However, the relation between reading comprehension and various dimensions of written composition did not differ by grade levels.

The findings suggest that reading-writing relations are not uniform and vary across different linguistic components and developmental stages. This meta-analysis provides a detailed and nuanced picture of these relationships.

 

Source: Kim, Y.-S. G., Wolters, A., & Lee, J. won. (2024). Reading and writing relations are not uniform: They differ by the linguistic grain size, developmental phase, and measurement. Review of Educational Research, 94(3), 311–342. https://doi.org/10.3102/00346543231178830… Read the rest

Categories
K-12 Education Social and Motivational Outcomes

Did interventions improve teacher well-being?

A meta-analysis by Li and colleagues investigated the effects of intervention programs on K-12 in-service teacher well-being. A total of 44 studies, comprising 176 effect sizes, published from 1990 to 2022 were included in the three-level meta-analysis. About 80% effect sizes was obtained from interventions with duration 8 weeks or less. Among these studies, 31 used a pre-test/post-test control group design, while 14 studies only compared pre/post intervention results. The overall mean effect size of 0.35.

The moderation analysis explored factors influencing effect variability across studies. The quality of each study was assessed based on five criteria (e.g., sampling method, study design), and were classified into strong, moderate, and weak. Analysis revealed that strong quality studies (g = 0.53, k=21) had the largest effect size, followed by weak (g=0.40, k=45) and moderate quality (g = 0.26, k =110) studies. Moreover, studies with a control group (g=0.39, k=111) had larger effects than those without (g=0.26, k=65), though not statistically significant.

Regarding intervention type, mindfulness-based interventions (g = 0.39, k = 79) had significantly larger effects than those targeting the mitigation of negative emotions (g = 0.15, k=13), but comparable to positive-psychology based interventions (g=0.38, k=29). While no significant differences were found, interventions had greater impact on personal well-being (e.g., life satisfaction, reducing stress) than occupational well-being (e.g., job satisfaction, occupational burnout).

The findings of the study may be inflated due to several limitations: 1) the study only included published peer-reviewed journal articles, excluding grey literature; 2) significant publication bias was found; and 3) all studies used self-report questionnaires for measurement.

Nevertheless, this meta-analysis provides a comprehensive and updated synthesis of the effects of interventions on teacher well-being.

 

Note: k = number of effect sizes.

 

Source: Li, Y., Wang, X., Chen, J., Lee, J. C.-K., Yan, Z., & Li, J.-B. (2024). The Intervention Effects on Teacher Well-being: A Three-Level Meta-Analysis. Educational Psychology Review, 36(4), 129. https://doi.org/10.1007/s10648-024-09966-y… Read the rest