Backes and Cowan (2018) have published a working paperon the differences between computer- and paper-based tests. In 2015, Massachusetts introduced the new PARCC assessment. School districts could choose whether to use the computer or paper versions of the test, and in 2015 and 2016, districts were divided fairly evenly between the two. The authors use this division to compare results for pupils in Grades 3–8 (Years 4–9). The analyses showed:
- Pupils who took the online version of PARCC scored about 0.10 standard deviations lower in Mathematics and about 0.25 standard deviations lower in English than pupils taking the paper version of the test.
- When pupils took computer tests again the following year, these differences reduced by about a third for Mathematics and by half for English.
The study also looked at whether the change to computer tests affected some pupils disproportionately, as the analyses showed:
- There were no differences for Mathematics, but for English there was more of an effect on pupils at the bottom of the achievement distribution, pupils with English as an additional language and special education pupils.
The authors point out that these differences not only have consequences for individual pupils, but for other decisions based on the data, including teacher and school performance measures and the analysis of schoolwide programmes.
Source (Open Access) : Backes, B. & Cowan, J. (2018). Is the pen mightier than the keyboard? The effect of online testing on measured student achievement (Working paper 190). National Center for Analysis of Longitudinal Data in Education Research. Retrieved from: https://caldercenter.org/sites/default/files/WP%20190.pdf