Category: Educational Administration and Leadership

Categories
Educational Administration and Leadership Kindergarten Primary School Education Programme Evaluation

Can we know the ingredients that make programs work?

Educational programs consist of several features that together have an impact on student achievement, such as the use of small group activities, or the focus on vocabulary enhancement. What is difficult to know is which components are the ones that make the program work and determine the size of this effect. This information would be of great relevance for researchers and program developers to design effective interventions.

The What Works Clearinghouse has applied a new meta-analysis method, called Bayesian meta-analysis, with the aim of exploring to what extent program components explain the impact of interventions. To investigate the potential of this new method, WWC used a total of 29 studies on the effects of 25 early literacy interventions in K–3 on alphabetics. A pool of experts in early literacy developed a taxonomy describing the program components and coded the 29 studies accordingly. The WWC focused its analysis on 15 component domains (e.g., instructional practices to build comprehension skills).

Overall, results showed that most of the multifaceted early literacy interventions (72%) have positive impacts on alphabetic outcomes. On the association between a specific component domain and intervention effects, results suggest that the use of ‘testing and screening’ (e.g., using formative assessment) and ‘student placement in groups based on assessments’ are the components with the larger impact, followed by ‘non-academic student supports’. Other domains, such as ‘educator support’ or ‘organizational structures’ were found to have a negative association. However, the analysis of component domains explained only 9% of the variation in the interventions’ effects. This means that 91% of the variation in effects is due to other components that were not considered: contextual factors, characteristics related to intervention implementation (e.g., duration), study quality, etc.

The authors declared that the statistical approach used is exploratory and practitioners should read results with caution. From a methodological perspective, future research is needed after this first trial on the association between components and programs’ effects with the methods proposed by the WWC. The exploratory results provide insights into the use of Bayesian meta-analysis and taxonomies in coding intervention components. Understanding why programs work is relevant to designing and implementing effective interventions. Evaluating which components are associated with large impacts is, however, challenging. Future research has to account for other factors (e.g., unmeasured components, duration, implementer, etc.). Additionally, coding program components is not always easy because the primary studies do not provide all relevant data. Unreported components may affect the accuracy of results.

 

Source (Open Access): Walsh, E., Deke, J., Robles, S., Streke, A., & Thal, D. (2023). Using Bayesian meta-analysis to explore the components of early literacy interventions (WWC 2023008). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.  https://ies.ed.gov/pubsearch/pubsinfo.asp?pubid=WWC2023008 … Read the rest

Categories
Educational Administration and Leadership K-12 Education Programme Evaluation

Unraveling the “winner’s curse”: Challenges and solutions for Clearinghouses

In recent research studies, concerns have arisen regarding the potential inflation of intervention effects in reported findings, a phenomenon known as the ‘winner’s curse’. This raises questions about whether clearinghouses inadvertently contribute to this issue by prioritizing studies with statistically significant results for replication or funding. The latest article by Valentine, who has experience as an investigator at the What Works Clearinghouse (WWC), explores the challenges faced by these organizations in this context.

Clearinghouses have the challenging task of evaluating the credibility of claims relevant to their consumers, distinguishing between more and less credible ones. This complexity arises due to the intricate nature of studies and the limited empirical evidence available regarding the importance of various aspects of study conduct. Despite these challenges, the author argues that establishing evaluation standards can benefit the research community by serving as a communication tool to emphasize crucial aspects of research design and implementation. The article addresses the limitations of clearinghouses in determining ‘what works’, because: (I) probability values from studies should not be directly interpreted as effectiveness, (II) study results are context-dependent, and (III) individual studies may not represent the broader body of research.

To mitigate the winner’s curse, two strategies are proposed for clearinghouses. Firstly, they should avoid presenting information in a way that may lead to misunderstandings; this could involve using Bayesian statistics to account for uncertainty in effect estimation. Secondly, the author suggests that clearinghouses should prioritize systematic reviews as their main mission, drawing attention away from individual studies and towards the body of credible research.

 

Source: Valentine, J. C. (2023). How evidence clearinghouses can avoid the winner’s curse. Journal of Research on Educational Effectiveness, 0(0), 1–5. https://doi.org/10.1080/19345747.2023.2194298… Read the rest

Categories
Achievement Educational Administration and Leadership Primary School Education Secondary School Education

Relation between parental homework involvement and students’ mathematics achievement

Informed by the self-determination theory, parental homework involvement has generally been measured by two dimensions: supportive parental homework involvement (SPI, e.g., autonomy support, provision of structure) and intrusive parental homework involvement (IPI, e.g., controlling, monitoring). A meta-analysis by Jiang and colleagues investigated the association between these involvement types and students’ mathematics achievement.

The meta-analysis identified 41 effect sizes from 20 empirical studies that reported Pearson’s correlation coefficient r and measuring tool of mathematics achievement. These studies were conducted from 2005 to 2022 and met the majority of the Joanna Briggs Institute Critical Appraisal Checklist criteria. Overall, for math achievement, a very small but significant positive effect was found for SPI (r = +0.076) in relation to math achievement, while a negative association was found for IPI (r = -0.153). Moderator analysis results are presented below.

  • Significant different effects were found among three sub-types of SPI, autonomy support showed the strongest link (r=+0.133), whereas content support (r=+0.049) and provision of structure (r=-0.009) displayed insignificant correlations.
  • Regarding IPI, no significant differences in effects were found between controlling (r=-0.154) and interference (r=-0.149).
  • Standardised math ability measurement (r=+0.036) showed a significantly weaker association with math achievement than non-standardised measures (r = +0.123) for SPI, but not for IPI.
  • The negative association between IPI and math performance was greater for middle school students (r = -0.228) compared to primary school students (r=-0.093). No significant difference was found among grade levels for SPI.
  • No significant difference was found among reporters (students versus parents) for either type of parental involvement.

The study highlighted that among reported sub-types of SPI, only autonomy support displayed a significant positive association with students’ math performance. Moreover, the negative association between IPI and math achievement was greater among higher grade-levels.

 

Source: Jiang, Q., Shi, L., Zheng, D., & Mao, W. (2023). Parental homework involvement and students’ mathematics achievement: A meta-analysis. Frontiers in Psychology, 14. https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1218534… Read the rest

Categories
Educational Administration and Leadership Effective Teaching Approach Secondary School Education

Teaching strategies for facilitating classroom discussion on citizenship in pre-vocational education

Many studies indicate that classroom discussions on citizenship topics play a significant role in citizenship education. These discussions not only influence individual citizenship outcomes and the classroom atmosphere, but also have implications for the democratic fabric of society in a wider societal context. However, training teachers in effectively guiding classroom discussions remains a challenging task, given the considerable variation in approaches to teaching citizenship education.

To address this challenge, Coopmans and Kan recently conducted an exploratory study to investigate pre-vocational teaching strategies to facilitate classroom discussions on citizenship. The authors compared teaching strategies employed by four pre-vocational teachers across three Dutch secondary schools during 26 lessons using the discussion tool “Terre Nova Mini Society” in the 2018/2019 school year. The three schools were selected due to their varying levels of familiarity with citizenship education and citizenship-related classroom discussions. Before teaching the lessons, the four teachers participated in a workshop to familiarize themselves with the lesson method and manual. They were then instructed to freely design their own lessons. Afterward, six researchers observed their lessons.

Based on the field notes from 26 classroom observations and teacher questionnaires, this study analyzed results from case reports, including qualitative and quantitative data. Results indicated that classrooms where teachers and students were less familiar with discussions seemed to benefit from structural aspects of discussions instead of strict content regulation. In contrast, classrooms more familiar with discussions seemed to gain most from reflection on both process and content, and incorporation of real-life consequences or current affairs. Given the teacher’s crucial yet complex role in facilitating classroom discussions conducive to citizenship education, it is a fruitful approach to combine student-regulated content guided by teacher guidance for a discussion’s structure.

 

Source: Coopmans, M., & Rinnooy Kan, W. F. (2023). Facilitating citizenship-related classroom discussion: Teaching strategies in pre-vocational education that allow for variation in familiarity with discussion. Teaching and Teacher Education, 133, 104268. https://doi.org/10.1016/j.tate.2023.104268… Read the rest

Categories
Educational Administration and Leadership K-12 Education

New national superintendent database illuminates trends in recent turnover by gender

Extensive teacher databases exist, yet no such database has recorded national data on school superintendents over multiple, consecutive years until now. A recent study by R. S. White documents how she collected data on superintendents of all K-12 public school districts in the U.S. between 2019 and 2023, manually verifying gender.  

White found that 74% of all U.S. superintendents were men in 2019-20 while 72% were men in 2022-23. In addition, 2019-20 saw a 14.2% superintendent turnover, while 2022-23 saw a 17.1% turnover, with 62% of districts experiencing no turnover in these four years. Yet underneath these seemingly minor changes were hidden some significant differences by gender.

Districts with a greater proportion of white students were more likely to have male superintendents. Of all districts led by men, turnover was more likely in those that had a greater proportion of students of color or students receiving free or reduced-price lunch (FRPL). On the other hand, districts with a greater proportion of FRPL students, students with individualized education plans, students who were learning English, or students of color were more likely to have female superintendents. Of all districts led by women, those more likely to experience turnover were those with greater proportions of white students and fewer FRPL students. Finally, districts with a high proportion of white students looking for a new superintendent were more likely to hire a man regardless of the gender of their prior leader, while districts with a high proportion of students of color and FRPL students were more likely to hire a woman superintendent for their empty seat.

More research exploring the relationship between characteristics of school superintendents with student outcomes, school climate, and educator effectiveness could prove valuable for better understanding the impact of the superintendency in this period of COVID recovery, racial justice reckoning, and culture wars.

 

Source: White, R. S. (2023). Ceilings made of glass and leaving en masse? Examining superintendent gender gaps and turnover over time across the United States. Educational Researcher, 52(5), 272–285. https://doi.org/10.3102/0013189X231163139… Read the rest

Categories
Educational Administration and Leadership Secondary School Education Social and Motivational Outcomes

Exploring the Mechanisms of Principal Instructional Leadership on Students’ School Happiness

In an effort to understand how principals and teachers can influence students’ sense of happiness at school, a study published in Educational Research of China National Academy of Educational Science analyzed a representative sample of teachers and students from 53 middle schools in a province in central China. The selected schools, located in four districts representing various socioeconomic conditions, encompassed diverse characteristics of schools and principals.

The mediating effects of all variables were: (1) Teachers’ sense of teaching efficacy mediated the relationship between the principals’ perceived instructional leadership and students’ school happiness (mediating effect = 12.6%); (2) Students’ perceived teacher-student relationships mediated the relationship between teachers’ sense of teaching efficacy and students’ school happiness (mediating effect = 34.9%); (3) Teachers’ sense of teaching efficacy and students’ perceived teacher-student relationship play a serial mediation role between principals’ perceived instructional leadership and students’ school happiness (mediating effect = 7.7%). The total mediating effects accounted for 55.2% of the variance, while the direct effect accounting for 44.8%.

The study identified three pathways in which principals’ instructional leadership influences students’ sense of school happiness: (1) The direct effect principals’ instructional leadership made on students’ school happiness; (2) Indirect effect via teachers’ sense of teaching efficacy; (3) Indirect effect by influencing teachers’ teaching efficacy, which in turn affects students’ perceived teacher-student relationships, ultimately influencing students’ school happiness.

To enhance students’ school happiness, it is necessary to improve principals’ instructional leadership skills, thereby stimulating teachers’ teaching efficacy and promoting the development of positive teacher-student relationships. These findings provide evidence-based mechanisms and pathways through which principals’ instructional leadership can influence students’ happiness.

 

Source: 陳霜葉, 榮佳妮, 郭少陽. 如何讓學生在學校感到幸福——校長教學領導力作用機制探索[J]. 教育研究, 2023, 44(02):88-100.… Read the rest