卓越實證概述 Best Evidence in Brief

Programme Evaluation

Effectiveness of a PD program in a repeated randomized study

A study published in Evaluation Review evaluated the effect of a teacher professional development program on student math achievement in two experiments. The evaluation was conducted through repeated randomized control trials on two consecutive cohorts of teachers. Since replication is rare in education, the authors wanted to evaluate the relevance of program implementation on its effectiveness. Two cohorts of 730 teachers and their 13,000 sixth grade students participated in the study in 2008 and 2010. In both cohorts, teachers were randomly assigned to the intervention or to the control group for three years of program implementation. The PD program, named PON M@t.abel+, was promoted by the Italian Ministry of Education to train math teachers in using strategies close to the students’ everyday life and learning by doing. Training was provided to groups of teachers (15-20 each group) by a tutor using both in-person and online sessions. The training covered different...

25 03 2022
Effectiveness of volunteer tutoring

Markovitz and colleagues recently reported on a replication and expansion of a previous randomized controlled trial focused on volunteer tutoring in reading for at-risk early elementary school students. The study focuses on the effectiveness of the Minnesota Reading Corps and the Wisconsin Reading Corps, which are both programs within AmeriCorps. The initial 2014 study focused solely on Minnesota and was limited in its ability to assess impacts for second and third grade students. The authors suggest the study is useful because aspects of the tutoring programs have changed, they are now evaluating the effects of tutoring in two separate programs, and they are now able to have a longer evaluation of the effects on second and third grade students. The study used a matched-pairs design in which students were matched based upon their baseline fall test scores, and then one student was assigned to the control group while the other...

25 03 2022
For whom does the Good Behavior Game work?

There are ample evidence that social-emotional learning programs support behavioral and academic outcomes in students. However, few studies have looked at the “who” and “why” that make these programs work. In a study published in the Journal of Consulting and Clinical Psychology, implementation variability and participant risk status were examined as predictors of disruptive behavior outcomes. In this large cluster randomized controlled trial, seventy-seven English primary schools (N = 3,084 children, aged 6–7) were assigned to either receive the Good Behavior Game or to continue with business-as-usual. The Good Behavior Game is a universal behavior management intervention that encourages students to monitor their behavior in return for tangible rewards. Due to the clustered nature of the data, hierarchical linear models were fitted to the data. This study used intent-to-treat as well as complier average causal effects samples to compare findings between the two. Interestingly, intent-to-treat analysis found no discernible impact...

11 03 2022
Does small group mathematics instruction work for all students?

A recent randomized evaluation conducted in Norway investigated the effects of a small group mathematics instruction on student academic achievement in elementary school. The study involved 159 elementary schools and 14,891 students in 10 municipalities in Norway. Schools were randomly assigned to receive the intervention or business as usual. Students in the intervention group were pulled out from their regular math class into small groups homogenous for ability for 3-4 hours per week. Therefore, the small group instruction was delivered to all students in the class, not only to low-achieving students or pupils at risk for failure. The intervention lasts for one or two years with two periods of 4-6 weeks of intervention each year. The authors evaluated the effects of the intervention on student mathematics achievement in fifth grade – after one or two years of intervention – through the Norwegian national test. Results showed significant positive effects for...

11 03 2022
Using songs to teach vocabulary

A recent study published in Early Childhood Research Quarterly by Lawson-Adams and colleagues explores the value of sung songs and rhythmically spoken songs in teaching vocabulary to preschool students. The researchers used a within-subjects design with a total of 56 students (mean age = 4.8 years old) from 4 preschool classrooms within a district. The intervention was administered 3 times to each class over the course of 2 weeks, with each intervention lasting approximately 15 minutes. Each intervention consisted of 3 activities: a picture-card only activity a picture-card plus sung song activity a picture-card plus rhythmically spoken song activity Each activity consisted of 6 targeted vocabulary words that were spoken by the teacher 4 times each during the activity. For the song activities, each word was spoken twice and then appeared in the song twice, while in the picture-card only activity each word was spoken 4 times. The students took...

25 02 2022
Parent engagement in science education: the more the merrier?

Parents are the first teachers of their children. The success of school education needs parental engagement. However, there is a cost to everything. What are the costs of encouraging more parental involvement in children’s science education? Robinson and her colleagues used a randomized field experiment to examine the effects of a text-messaging intervention in science education and identified opportunity costs of shifting parental effort from other subjects to science. The study took place in England. Grades 7-11 students from 5 secondary schools were randomly assigned to the treatment condition (n = 1,729) or the control condition (n = 1,754). Pre-treatment covariates showed no statistically significant differences between conditions. Through a pre-existing texting platform, parents of students in the treatment condition received around two text messages per week nudging them to ask students specific questions tied to their science curriculum. All else equal, parents in the control condition did not receive...

25 02 2022
Effectiveness of BARR for students

A recent randomized evaluation conducted by Borman and colleagues investigated the effects of Building Assets, Reducing Risks (BARR) on student experience and academic outcomes. BARR is a model developed to address barriers to student success and consists of different strategies to build and improve relationships between staff, staff and students, and between students. Teachers and administrators in schools receive initial training and on-going coaching through observations and feedback on implementation. The study involved about 3,000 ninth graders from 11 American schools randomly assigned to receive the intervention over one year or to continue with their regular practice. Student academic achievement was measured through a standardized test, the Northwest Evaluation Association’s (NWEA) Measures of Academic Progress (MAP). Student experience was evaluated only at posttest. Results showed significant positive effects for mathematics achievement (ES = +0.11) but no significant results in reading (ES = +0.06). The authors concluded that BARR is a...

25 02 2022
Short-term and long-term effects of the School Improvement Grants initiative

School Improvement Grants (SIGs) were grants for state education agencies to address underperformance in public schools in the US. The SIG program required schools to adopt a reform model by choosing among four alternatives: the transformation model required reforms in the school instructional and evaluation system and changing the leadership; the turnaround model required the same transformations plus replacing 50% of the staff; the restart model required closing the school and opening it under the leadership of an education management organization; and the closure model required closing the school. Among these models, most of the SIG schools chose the transformation model, some schools the turnaround model, and a few schools the restart model. A recent study evaluated the effects of SIGs on student academic achievement and graduation rates, focusing on schools that adopted the transformation and turnaround models. A total of 99 schools and 35,200 students in grades 3 and...

11 02 2022
Does technology work to improve English Learners outcomes?

Rosetta Stone Foundations developed a software to improve English learners’ outcomes through individualized support and practice. The software, called Rosetta Stone Foundations English (RS), is a self-paced tool to be used as supplemental material to regular instruction in class. Its primary focus is on oral language skills by including speaking and listening practice. Students receive feedback on speaking accuracy as well as on writing, reading, and listening activities. A randomized study evaluated the effects of the program on English learners’ outcomes in grades 6-8. Eight schools with 152 students were randomly assigned to the treatment and control conditions. Students worked with the program for 32 weeks for an average of 90 minutes per week. Student proficiency in language was assessed using the TELL (Test of English Language Learning) diagnostic test that included several measures: Speaking-Listening TELL Composite, Reading-Writing TELL Composite, Reading-Aloud TELL Composite. For Speaking-Listening TELL Composite, positive effect was...

21 01 2022