How Reading Recovery probably worksPosted: July 29, 2016
I have written before about trials of Reading Recovery, particularly the recent I3 study from the U.S. Since then, I have become aware of two papers that I think are key to understanding the way that Reading Recovery works.
To say that it ‘works’ is actually quite controversial. Objectively, it does. Placing students in a Reading Recovery intervention seems to improve their reading more than if you don’t do anything. The question remains as to why this is the case. For instance, is it due to the specialist training that Reading Recovery teachers receive?
It is important to note that Reading Recovery is a one-to-one intervention of up to 60 half-hour sessions. This is hugely resource intensive. It also represents Benjamin Bloom’s ideal of a maximal form of teaching. He reviewed various interventions – specifically conventional teaching, mastery learning and tutoring – and found an effect size of d=2.0 for one-to-one tutoring. So the form of Reading Recovery likely contributes some proportion of its effect.
We could possible gauge this by comparing Reading Recovery directly with another one-to-one reading intervention of the same duration and randomising students between the two treatments. Surprisingly, there seem to be few such direct comparisons. So perhaps we should look at comparing effect sizes from Reading Recovery versus a control with effect sizes from rival one-to-one programs versus a control. This is more fraught because conditions will necessarily vary but it might be indicative.
This is where the second paper comes in. In 2011, Robert Slavin and colleagues reviewed a number of studies on reading interventions. They were quite picky about the studies that they included. When it came to Reading Recovery, they avoided outcome measures that were intrinsic to the method itself in favour of more objective measures:
“First, most Reading Recovery studies use as posttests measures from Clay’s (1985) Diagnostic Observation Survey. Given particular emphasis is a measure called Text Reading Level, in which children are asked to read aloud from leveled readers, while testers (usually other Reading Recovery teachers) record accuracy using a running record. Unfortunately, this and other Diagnostic Observation Survey measures are closely aligned to skills taught in Reading Recovery and are considered inherent to the treatment; empirically, effect sizes on these measures are typically much greater than those on treatment-independent measures.” [my emphasis]
At this point I will remind you of my first principle of educational psychology: students tend to learn the things you teach them and don’t tend to the learn the things you don’t teach them.
Slavin et. al. also ruled-out studies based only upon those students who had successfully completed Reading Recovery. Such studies prove little. I am sure that many teachers would prefer to be judged only on the results of those students who have been successful.
Once they had whittled-down the research in this way, Slavin et. al. were able to note that:
“The outcomes for Reading Recovery were positive, but less so than might have been expected…
Across all studies of one-to-one tutoring by teachers, there were 20 qualifying studies (including 5 randomized and 3 randomized quasi-experiments). The overall weighted mean effect size was +0.39. Eight of these, with a weighted mean effect size of +0.23, evaluated Reading Recovery. Twelve studies evaluated a variety of other one-to-one approaches, and found a weighted mean effect size of +0.56…
Across all categories of programs, almost all successful programs have a strong emphasis on phonics. As noted earlier, one-to-one tutoring programs in which teachers were the tutors had a much more positive weighted mean effect size if they had a strong phonetic emphasis (mean ES = +0.62 in 10 studies). One-to-one tutoring programs with less of an emphasis on phonics, specifically Reading Recovery and TEACH, had a weighted mean effect size of +0.23. Within-study comparisons support the same conclusion. Averaging across five within-study comparisons, the mean difference was +0.18 favoring approaches with a phonics emphasis.”
I think it is important that policymakers are aware of these findings.