Category: Educational Administration and Leadership

Categories
Achievement Educational Administration and Leadership K-12 Education

Effectiveness of collaborative teaching models on academic achievement

A recent meta-analysis published in Review of Educational Research examined the effects of co-teaching. Co-teaching and related collaborative teaching models are defined as interventions delivered by two or more teachers or professionals in the same physical space. The authors included randomized and quasi-experimental studies that investigated the effects of collaborative teaching models on student outcomes in arts, social science, and STEM subjects in K-12, published between 1984 and 2020. After excluding studies that did not meet the criteria, as well as another 52 studies for low methodological quality, 76 studies were included in the review.

The mean effect size across the studies was +0.11, with a large variability. Characteristics highlighted in the literature as relevant factors (e.g., duration, teacher training) for the impact of co-teaching did not play a role in the results. Similar results were found for interventions conducted by trained teachers and by teaching assistants without a qualification, suggesting that co-teaching may be implemented in a more cost-effective way.

 

Source: Vembye, M. H., Weiss, F., & Hamilton Bhat, B. (2024). The effects of co-teaching and related collaborative models of instruction on student achievement: A systematic review and meta-analysis. Review of Educational Research, 94(3), 376–422. https://doi.org/10.3102/00346543231186588… Read the rest

Categories
Educational Administration and Leadership Higher Education Secondary School Education

How test anxiety related to test performance through self-test practice?

Previous research has demonstrated that practice testing, or practice retrieval, is an effective learning technique. This method can be implemented in two ways: either through mandatory tests like class quizzes, or through voluntary self-testing without formal instruction. Regarding the latter, Liu and colleagues conducted a study to explore whether test anxiety and test stakes affect self-testing usage and, in turn, influence learning performance.

The study included one survey and four experiments. Data for the survey study were collected from 459 Grade 11 students at a school in northwestern China. It used a self-report questionnaire to measure trait test anxiety and one item to ask self-testing frequency. Academic performance was accessed with school exams results. Mediation analysis indicated that students with high test anxiety not only directly lowered exam scores but also indirectly did so through reducing the frequency of self-testing.

In laboratory experiments, participants underwent three stages: learning material, selecting a revision strategy (restudying or self-testing), and taking a test. Conducted at Beijing Normal University, the first experiment with 70 undergraduates showed that high trait test anxiety reduced the likelihood of choosing self-testing. The second experiment (n=25) revealed that self-testing was less common for high-stakes than low-stakes tests. The third (n=36) demonstrated that high stakes significantly impaired test performance by reducing self-testing.

The final quasi-experiment combined measures of test stake, testing anxiety, self-testing usage, and test performance. It randomly assigned two classes of students at Tianjin Vocational Institute to high- or low-stakes test conditions. State test anxiety was measured three time points: at the beginning of the experiment, after students were informed of the test stake condition, and after the study stage. Statistics revealed that the increase of state anxiety was higher in the high-stakes group compared to low-stakes group following test stakes manipulation. Chain mediation analysis showed that high stakes increased state anxiety, decreased self-testing selection, and subsequently hindered test performance.

This study supports the authors’ proposed mechanism whereby test anxiety negative impacted learning performance through discouraging of self-testing practice.

Source: Liu, S., Zhao, W., Shanks, D. R., Hu, X., Luo, L., & Yang, C. (2024). Effects of test anxiety on self-testing and learning performance. Educational Psychology Review, 36(2), 59. https://doi.org/10.1007/s10648-024-09882-1… Read the rest

Categories
Educational Administration and Leadership Secondary School Education

The bird of time is on the wing in U.S. schools

Policymakers have noted that because U.S. schools are not in session as long as schools in other countries, and because the time U.S. students spend in school is not always used efficiently, American students are placed at a disadvantage compared to their peers worldwide. Compounded by the impact of the Covid-19 school closures, some districts have increased the school year by 4-5 weeks to compensate, while other districts who continue to experience staffing shortages and reduced budgets have had to adopt a 4-day school week in response.

In a comprehensive analysis, Kraft and Novicoff explored the obstacles to active learning time, reviewed evidence on how time impacts student achievement, and examined data from district, national, and international sources. Their findings highlighted a positive correlation between time spent in school and student achievement. Notably, international studies showed significant gains when school hours were substantially increased (e.g., by 3 hours), though even smaller increases (less than 90 minutes) led to modest improvements.

The study also found that four-day school weeks rarely enhanced academic outcomes, whereas later school start times often did. Obstacles to active learning included unexcused student absences, outside interruptions, and teacher absences. The authors examined one district as a case study and found that high school students there lost 25% of their instructional time due to these factors. An increase of 1.85 hours a day would be needed to make up for this loss.

Kraft and Novicoff also addressed equity and extended learning time, highlighting that while extended learning can benefit students who have access to it, caution is needed to ensure it does not constrain valuable experiences outside of school. Policy implications include the need for equitable access, balanced scheduling to accommodate diverse student needs, and ensuring quality instruction during extended hours.

Source: Kraft, M. A., & Novicoff, S. (2024). Time in school: A conceptual framework, synthesis of the causal research, and empirical exploration. American Educational Research Journal, 00028312241251857. https://doi.org/10.3102/00028312241251857… Read the rest

Categories
Educational Administration and Leadership K-12 Education Language Development Programme Evaluation

The effects of professional development on student and teacher outcomes

Effective instruction by well-prepared teachers is crucial for enhancing students’ reading comprehension. This process begins with high-quality professional development (PD) that enriches teachers’ knowledge and improves their instructional strategies in reading comprehension.

A recent meta-analysis reviewed 29 experimental or quasi-experimental studies focused on PD for reading comprehension among teachers. It examined outcomes for both teachers and K-12 students to assess which PD components influenced teacher knowledge, instructional practices, and student reading comprehension. Utilizing random-effects robust variance estimation, the analysis identified significant positive effects of PD on teacher outcomes (ES = +0.95) and smaller yet still significant effects on student comprehension (ES = +0.19).

The study found that specific PD characteristics such as coaching, format, content, providers, duration, and number of strategies did not significantly moderate teacher outcomes. Similarly, study quality characteristics (e.g., design, control group condition, outcome measure) did not show significant moderation effects, though distinctions between standardized and researcher-created measures were noted (ES=+0.11 vs +0.44).

A notable conclusion from this study is that future research should prioritize direct testing of effective PD strategies. While the findings of this meta-analysis align with previous research, there are potential limitations in statistical power arising from substantial heterogeneity among studies. There is also a possibility that PD characteristics interact in complex ways. In addition, the notable overall positive effect on teacher outcomes should be approached with care, as a majority of measures relied on teacher observations or self-reports.

Source (Open Access): Rice, M., Lambright, K., & Wijekumar, K. (Kay). (n.d.). Professional development in reading comprehension: A meta-analysis of the effects on teachers and students. Reading Research Quarterly, n/a(n/a). https://doi.org/10.1002/rrq.546… Read the rest

Categories
Educational Administration and Leadership K-12 Education Programme Evaluation

The evidence on high-dosage tutoring, and proposed metrics for evaluating efficiency and cost

The national nonprofit Accelerate released a new report that underscores the substantial impact of high-dosage tutoring while acknowledging that the effectiveness of tutoring programs varies significantly, necessitating deeper exploration. The report seeks to further contextualize the impact of tutoring on student learning, introduces novel metrics for evaluating tutoring programs’ efficiency and cost-effectiveness, and proposes a research agenda for future studies.

Researchers Kohlmoos and Steinberg evaluated existing evidence on the impact of high-dosage tutoring, identifying RCTs with at least 350 participants from Nickow et al.’s 2023 meta-analysis of tutoring program impacts, while also including additional studies that met rigorous standards. Notably, they excluded researcher-made measures from the assessment process, focusing solely on well-designed RCTs with standardized assessments – resulting in 15 studies of 12 tutoring programs. This emphasis on standardized measures represents a significant advancement in the evidence movement, enhancing the rigor and reliability of the evidence base. This shift ensures consistency and comparability across studies while promoting transparency and replicability, ultimately strengthening the credibility of educational research findings (see more on this topic in BEIB here and here).

The report also establishes measures of tutoring efficiency (hours needed to improve student learning by one month) and cost-effectiveness (additional months of student learning produced at a cost of $1,000 per pupil). These metrics aim to guide future research in comparing the return on investment across different tutoring programs, assisting schools in evidence-informed decision-making when selecting providers. By normalizing program impact by both dosage and cost, the report provides valuable insights for comparing different tutoring programs and guiding resource allocation decisions.

Additional recommendations for researchers, tutoring providers, federal agencies, states, and school districts included prioritizing large-sample evaluations to enhance evidence on tutoring effectiveness, advocating for transparent data reporting from providers, and establishing state-curated lists of evidence-based tutoring programs. Ultimately, the report seeks to shift the conversation around tutoring toward empirically-informed discussions on program models that offer optimal resource utilization, paving the way for more effective and efficient educational interventions.

Source (Open Access): Kohlmoos, L., & Steinberg, M. P. (2024). Contextualizing the impact of tutoring on student learning: Efficiency, cost effectiveness, and the known unknowns. Accelerate, The National Collaborative for Accelerated Learning. https://accelerate.us/wp-content/uploads/2024/05/Accelerate-Research-Report-Efficiency-and-Cost-Effectiveness-1.pdf… Read the rest

Categories
Educational Administration and Leadership K-12 Education

Risk of bias in education systematic reviews: Are published reviews of good quality?

Educational researchers are increasingly concerned with the quality of systematic reviews because review results play a crucial role in informing policy and practice decisions. A recent study published by Nordström and colleagues assessed the risk of bias and the open science practice of education systematic reviews.

The study evaluated 88 reviews published between 2019 and 2021 and focused on the impact of K-12 interventions on student outcomes. The ROBIS (Risk Of Bias In Systematic reviews) scale was used to assess the quality and reliability of systematic reviews. The scale consists of four domains: eligibility criteria, identification and selection of the studies, data collection and study appraisal, and synthesis and findings. Practices of open science, such as protocol preregistration and availability of data, were also assessed.

Results showed that 88% of the reviews had a high risk of bias. Among them, thirty-one reviews did not conduct a comprehensive and reproducible search. About 30 reviews did not formally evaluate the risk of bias of the original studies. A small portion of the assessed reviews (11%) were judged to have a low risk of bias. Of these, only six reviews pre-registered a protocol and three shared their data.

The authors concluded that it is crucial to implement all aspects of systematic review reporting in future research to ensure replicability. Open science has been also identified as an area needing improvement.

Source (Open Access): Nordström, T., Kalmendal, A., & Batinovic, L. (2023). Risk of bias and open science practices in systematic reviews of educational effectiveness: A meta-review. Review of Education, 11(3), e3443. https://doi.org/10.1002/rev3.3443… Read the rest