Category: Educational Administration and Leadership

Categories
Educational Administration and Leadership Primary School Education Secondary School Education

Surprise rewards for good attendance had a surprising consequence

A working paper by Carly Robinson and colleagues, published by the Harvard Kennedy School, reports on an experiment to measure the impact of attendance rewards on students.

The trial included 15,629 sixth through twelfth grade students from 14 school districts in California. All the students had previously had perfect attendance in at least one month in the fall. The students were randomly allocated to one of three groups:

  • “Prospective Award” students received a letter telling them they would receive a certificate if they achieved perfect attendance in February (the following month).
  • “Retrospective Award” students received a letter and certificate telling them they had earned an award for perfect attendance during one month in the fall term.
  • Control students received no communication.

The researchers collected data on the students’ attendance in the following month (February). They found that:

  • There was no impact of offering the prospective reward on subsequent attendance.
  • Offering the retrospective award resulted in students attending less school in February. Absences among this group increased by 8% (an average of 0.06 days per student).

The researchers suggest that the retrospective awards may have sent unintended signals to the students, telling them that they were performing better than the descriptive social norm of their peers, and exceeding the institutional expectations for the awarded behavior.

 

Source (Open Access): Robinson, C.D., Gallus, J., Lee, M.G. & Rogers, T. (2018). The demotivating effect (and unintended Message) of retrospective Awards – HKS faculty research working paper series RWP18-020. Retrieved from https://research.hks.harvard.edu/publications/getFile.aspx?Id=1681… Read the rest

Categories
Educational Administration and Leadership Programme Evaluation Secondary School Education

Grouping students by achievement

The Education Endowment Foundation in the UK has published an evaluation of two trials of programs developed by the University College-London (UCL) Institute of Education investigating approaches to grouping students: Best Practice in Setting and Best Practice in Mixed Attainment Grouping.

The main trial, “Best Practice in Setting,” tested an intervention that aimed to get schools to improve their setting practice (grouping students in classes by their current achievement levels). A total of 127 schools took part in the trial, which ran over the course of two academic years. Teachers were randomly allocated to sets to prevent “lower” sets from being disproportionately assigned less-experienced teachers, while students in Years 7 and 8 were assigned to sets based on independent measures of achievement, rather than more subjective judgments such as behavior and peer interactions. There were opportunities throughout the year to re-assign students to different sets based on their current level of achievement. The evaluation showed:

  • No evidence was found that the intervention improves outcomes in math (effect size = -0.01) or English (effect size = -0.08).
  • Also, no conclusive evidence was found that the intervention improves students’ self-confidence in either subject.
  • The process evaluation revealed mixed views from participants, and many interviewees thought that what they were being asked to do represented little change from what they already do.

The researchers noted that because school and teacher buy-in was low and attrition rates for follow-up testing were high, half of the schools in the math trial and more than half of the schools in the English trial stopped the intervention before follow-up, and this makes it difficult to conclude anything certain about the impact of Best Practice in Setting.

 

Source (Open Access):Roy,P., Styles,B., Walker,M., Morrison,J., Nelson, J. & Kettlewell, K. (2018). Best Practice in Grouping Students Intervention A: Best Practice in Setting : Evaluation report and executive summary. London: The Education Endowment Foundation.

Source (Open Access):Roy,P., Styles,B., Walker,M., Morrison,J., Nelson, J. & Kettlewell, K. (2018). Best Practice in Grouping Students Intervention B: Mixed Attainment Grouping :Pilot report and executive summary. London: The Education Endowment Foundation. … Read the rest

Categories
Educational Administration and Leadership Kindergarten Primary School Education Programme Evaluation Secondary School Education

Reviewing the evidence on Congressionally-authorized federal programs

As part of their Straight Talk on Evidence initiative, the Laura and John Arnold Foundation has released a new report titled, “When Congressionally-authorized federal programs are evaluated in randomized controlled trials, most fall short. Reform is needed.” The report reviews research from 13 randomized controlled trials (RCTs) of Congressionally-authorized federal programs, such as Head Start, Job Corps, Abstinence Education, and Washington, D.C. school vouchers.

According to the report:

  • Eleven of the 13 RCTs found that the programs produced either no significant positive effects on the key targeted outcomes or small positive effects that dissipated shortly after participants completed the program. One RCT found sizable program effects (the Defense Department’s National Guard Youth ChalleNGe program) and another found modest effects (the Department of Labor’s Job Training Partnership Act).
  • However, among the 11 disappointing findings, evidence suggests that a subset of activities funded by these programs were indeed effective. Thus, reforming the programs to incorporate evidence-based funding criteria could lead to much better results.

The report says, “In a world where most attempts to make progress fail and a few succeed, spending as usual without a clear focus on evidence about what works is unlikely to solve the nation’s problems.” The report also provides a link to the final report from each RCT.

 

Source (Open Access): Straight Talk on Evidence. (2018). When Congressionally-authorized federal programs are evaluated in randomized controlled trials, most fall short. Reform is needed. Retrieved from http://www.straighttalkonevidence.org/2018/06/13/when-congressionally-authorized-federal-programs-are-evaluated-in-randomized-controlled-trials-most-fall-short-reform-is-needed/… Read the rest

Categories
Educational Administration and Leadership Kindergarten Primary School Education Secondary School Education Uncategorized

How much does education improve intelligence?

A recent meta-analysis published in Psychological Science looks at how much education improves intelligence, and suggests that a year of school improves pupils’ IQ scores by between one and five points.

Ritchie and his colleagues looked at three particular types of quasi-experimental studies of educational effects on intelligence:

  1. Those estimating education-intelligence associations after controlling for earlier intelligence.
  2. Those using policy changes that result in individuals staying in schools for different lengths of time.
  3. Those using school-entry age cut-offs to compare children who are similar in age but who have different levels of schooling as a result of their specific birth dates.

Their meta-analysis comprised 142 effect sizes from 42 data sets involving over 600,000 participants.

  • All three study designs showed consistent evidence that the length of time spent in school is associated with increased intelligence test scores (an average effect of +3.4 IQ points for one additional year of education).
  • The third study design, age cut-off, had the largest effect size (+5.2 IQ points).
  • The first study design showed the lowest effect (+1.2 IQ points).
  • For policy change, the effect size was +1 IQ point.

The authors suggested that education seems to be the most consistent, robust, and durable way that has been identified for raising intelligence.

 

Source: Ritchie, S. J., & Tucker-Drob, E. M. (2018.). How much does education improve intelligence? A meta-analysis, Psychological Science. Advanced online publication. doi:10.1177/0956797618774253.… Read the rest

Categories
Educational Administration and Leadership Kindergarten

The impact of professional development in early childhood education

Franziska Egert and colleagues in Germany and Amsterdam have conducted a review of the effects of professional development (PD) for early childhood educators on programme quality and children’s educational outcomes.

Studies were only included if they addressed quality of child care or child development, included early childhood teachers (including preschool, kindergarten and centre-based care), were quantitative, were experimental or quasi-experimental, reported effect sizes or data and addressed children 0–7 years old. This yielded 36 studies of 42 programmes evaluating quality ratings, and 9 studies of 10 programmes evaluating both quality ratings and pupil outcomes.

Results showed that:

  • Professional development improved the external quality ratings (as evaluated using the Classroom Assessment Scoring System, Early Language and Literacy Classroom Observation, Environmental Rating Scales and Individualized Classroom Assessment Scoring System) of early childhood education (effect size=+0.68)
  • Programmes providing 45–60 PD hours having the greatest impact on classroom practice as compared to programmes offering fewer or more hours. This was true regardless of whether teachers held a university degree or not.
  • Further, programmes that solely used coaching were almost three times as effective as other programmes.

A second meta-analysis of a subset of studies (n=486 teachers, 4,504 children) showed that improvement in the quality of early childhood education programmes was correlated with improvements in child development (effect size=+0.14) as determined by language and literacy scores, math scores, social-behavioural ratings, and assessment of cognition, knowledge and school readiness.

Source: Egert, F., Fukkink, R. G., & Eckhardt, A. G. (2018). Impact of in-service professional development programs for early childhood teachers on quality ratings and child outcomes: A meta-analysis. Review of Educational Research, 88(3), 401–433. … Read the rest

Categories
Educational Administration and Leadership Kindergarten Primary School Education Secondary School Education Social and Motivational Outcomes

Preventing and addressing behaviour problems

The What Works Clearinghouse (WWC) has posted a tip sheet with five evidence-based strategies to help educators prevent and address behaviour problems. These strategies, which are based on reviews of research and recommendations from experts in the field, are as follows:

  1. Modify the classroom environment to alter or remove factors that trigger problem behaviours (eg, revisit and reinforce expectations, modify the learning space to motivate pupils, and vary teaching strategies to increase academic success).
  2. Identify, deliver, and reinforce explicit teaching in appropriate behaviour.
  3. Learn about interventions that can help support pupils with an emotional/behavioural problem in making good choices. The WWC has identified effective interventions.
  4. Adapt teaching to maintain or increase pupil engagement in academics, preventing disruptive behaviour. The WWC offers strategies to engage pupils in reading, writing, maths, and out-of-school-time learning.
  5. Enlist adult advocates to help pupils at risk of dropping out address academic and social needs.

Better: Evidence-based Education magazine has addressed similar topics in classroom management and social-emotional learning.

 

Source (Open Access): The What Works Clearinghouse. (2014). Preventing and addressing behavior problems. Retrieved from https://ies.ed.gov/ncee/wwc/AddressingBehaviorProblems… Read the rest