Few educational interventions have generated as many studies as Reading Recovery.
Reading Recovery involves giving struggling readers one-to-one tuition. I have no doubt that it has an effect, as any one-to-one intervention would have an effect. Some people claim that no other intervention has as much evidence of effectiveness at scale. That may be true, but that is likely to be because no other intervention has been tested so much at scale.
When I suggest that it has an effect, I need to be careful about what I mean. Most studies compare Reading Recovery to doing nothing and the students in the ‘do nothing’ control group have usually had a diet of so-called ‘balanced literacy‘ – the whole-language wolf hiding in the Grandma’s bed of phonics. I have written before that we can’t be sure why Reading Recovery works. It could be the proprietary techniques taught to Reading Recovery teachers or it could just be the effect of one-to-one tuition. Indeed, other forms of one-to-one tuition seem to be more effective than Reading Recovery.
It would therefore be really useful if an organisation such as the Education Endowment Foundation used its extensive taxpayer funded resources to run a three-armed randomised controlled trial testing Reading Recovery against a one-to-one systematic synthetic phonics programme and against a control. My prediction would be that both interventions would outperform the control but the phonics intervention would outperform Reading Recovery. Unfortunately, the Education Endowment Foundation seem more interested in kooky Philosophy for Children and in trying to prove the evils of ability grouping.
In the meantime, we can expect to see more studies like a new one conducted on behalf of the KPMG Foundation. On the surface, the results of Reading Recovery seem extraordinary. For instance:
“49% of the Reading Recovery group achieved the nationally expected level of qualification for educational progression (5 or more GCSEs at the former A* to C grades, including English and Maths, equivalent to grades 8 to 4 in the current system), compared to a national average of 54% for all pupils in the same year. Only 23% of the comparison group reached this level.”
Although extraordinary, it is plausible that an early reading intervention could have such a profound effect. After all, academic learning relies heavily on reading. It is a foundational skill. Unfortunately, the study does not provide evidence to justify such a conclusion because of the way it was designed. It was not a randomised controlled trial and it was not even a good example of a quasi-experimental study.
Researchers identified 148 struggling readers in schools that did not offer Reading Recovery and 145 struggling readers in schools that did offer it. Teachers then selected just under two thirds of the students (91) in the Reading Recovery schools to give the intervention. They then compared the results of these 91 students with the 148 in the comparison schools. Can you see the problem?
You either need to compare the results of all 145 of the initially identified students with the 148 in the control, or you need to select roughly two thirds of the control using the same criteria with which you selected the Reading Recovery students and compare this cohort with the 91 students who had the intervention.
The authors claim that the 91 students who were selected were those who were the most in need of the intervention:
“It was not possible to offer Reading Recovery to all the children in Reading Recovery schools. Of the 145 children in Reading Recovery schools, 91 received Reading Recovery (though not all were successfully discontinued), 54 did not. The selection of children to receive Reading Recovery is made by the teacher and teacher leader, informed by children’s performance on the assessments and on age (the lowest achieving children are prioritised, and older children often taken first).”
If this is true then including the additional 54 students would likely only improve the GCSE results for this cohort. On the other hand, if the addition of these extra 54 students washed out any gains, we might conclude that Reading Recovery has no net effect. After all, there is nothing wrong with an intervention which only works with a proportion of the students, but we need to know if this is the case or whether it is actually a zero-sum game. And there is reason to believe that it might be. There are those who suggest, for instance, that contrary to the claims made about selection in this report, it is the more able students who often end-up in a Reading Recovery intervention:
None of this speculation would be relevant if we could see data for all 145 students rather than 91, but we cannot.
A fundamental principle of science is that we compare like with like. There is a hint in the data that the 91 Reading Recovery students were also more affluent than the 148 comparison group. Only 43% qualified for free school meals compared with 62% in the comparison group. The researchers tried to control for this by running separate statistical tests on the free-school-meal and non-free-school-meal populations of each sample i.e. comparing free-school-meal Reading Recovery students with free-school-meal control students. I’m not sure that quite solves the problem of the mismatched groups because wealth is a continuous variable whereas access to free-school-meals is a binary (i.e. the free-school-meal students in the comparison group could still be less affluent, on average, than the free-school-meal students in the Reading Recovery group). There were also slightly more boys in the control group (65% versus 60%). Clearly, socioeconomic status and gender impact on reading outcomes and so they may have been a factor here. These problems would have been avoided by randomisation.
There is also some interesting data on special educational needs that is presented as part of the results of the study. For instance, at age 14, “There were significantly fewer Reading Recovery pupils with a SEN status (35%) than comparison group pupils (52%).” We are invited to believe that this was an effect of the Reading Recovery intervention, but it may actually represent an underlying difference between the populations in the intervention and control groups. Many special educational needs take time to be identified and these may well have been present and unidentified or latent at the time of the initial allocation of students to groups.
In short, this new study demonstrates nothing much, even if we are inclined to believe that Reading Recovery has some effect.
The reason it is necessary to critique studies of this kind is that there are so many of them. As they pile up, commentators make statements to the effect that no other reading intervention has generated such a wealth of positive evidence and the individual studies get buried behind Hattie- or Education Endowment Foundation-style ‘effect sizes’ that teachers and school leaders take as evidence of effectiveness.
But it is not evidence. It is a house of cards.
Filling the pail 
KPMG have form. In February 2009 Policy Exchange published our report on Every Child A Reader, the main vehicle for funding Reading Recovery in England under New Labour. We reported that KPMG’s analysis claimed that
“…by the age of 37, each illiterate pupil will have cost the taxpayer an additional £42,000. However, these savings are hypothetical—and the costs are not. KPMG’s report is strikingly at odds with an earlier American study which estimated that Reading Recovery returned thirty cents on the dollar, at best.”
The report took into consideration the cost of obesity to the taxpayer, yet of course there was no evidence whatever that RR kept children from getting fat. A similar howler was their claim that the programme cost ‘only’ £2,400 per child. This was just the cost to the school, and it ignored both the massive cost of supporting and training RR teachers and the fact that one in five RR pupils were ‘referred on’ for additional support. We found that the true cost was £6,000 for each ‘successful’ intervention, and of course there was no objective criteria to say just what consituted a ‘successful’ intervention.
One of the little ironies here is that Nick Gibb worked for KPMG before entering politics, yet once he became Schools Minister, ECAR was cancelled. Politcally, it was an easy decision: the schools we talked to were only too aware of what they could do with the money that was squandered on RR.
An excellent response to an excellent post!
It might be pertinent to note that the children in this study almost inevitably received the old National Literacy Strategy ‘Searchlights’ instruction when they were first taught to read*. Since then, the National curriculum has been radically changed to mandate SSP as the initial teaching ‘method’ and the Phonics Screening Check has been introduced. These initiatives may well have led to a reduction in the number of children leaving Key Stage 2 with poor or non-existent reading skills. At the time of the ECAR (i.e RR) initiative some 20% of children were leaving KS2 with level 3 or below in English. As a reading tutor working with struggling readers at KS3 I absolutely know that L3 did not indicate any reading competence.
I would be interested to know if that figure of 20% is still applicable. If it is now lower it looks very much as if RR are trying to push their outdated intervention on a school population which may no longer have such an uergent need for their intervention.
It also strikes me that IF the figure is lower it probably encompasses the 20%+ of children which RR routinely ‘discontinues’ because they are unable to succeed with them.
*No studies ever seem to consider the method by which children are initially taught reading.
Pingback: Another flawed Reading Recovery study to add to the pack – Filling the pail – The Literacy Echo Chamber
I was hoping you might do a post about this latest RR report, Greg, thank you very much. I’m relieved you are so quick off the mark because it’s enabled me to add your comments to further reviews of RR. As you know, people have been drawing attention to their worries about RR and its inadvisable multi-cueing word-guessing for some considerable time – and these reviews are from across the world. The current news about RR in England feels like Groundhog Day considering we managed to achieve research-informed systematic synthetic phonics embedded in England’s statutory National Curriculum. If you mix SSP with RR, however, you undermine the phonics provision and there is a danger of setting children up on short and long term very poor reading habits. See here:
https://iferi.org/iferi_forum/viewtopic.php?f=3&t=1157&sid=663e0cf8c5d529c7c9b7a2b85ffa7097
Pingback: The Reading Recovery moral dilemma – Filling the pail
Reblogged this on The Echo Chamber.
The studies should also include the financial and time cost of teacher training. Just think if teachers learned much of this pre-service! @ReadOrPrison on Twitter.
Pingback: The Key Features of Remedial Education (Updated 2019)
As mentioned in this,researches show that many students in schools struggle to read. Still many fail to provide reading recovery to such students. A remedial reading program can help them increase their decoding skills and phonological awareness. Studies prove that around 31% of the students who enter kindergarten with ADHD will have difficulty learning to read. In such cases, a remedial reading program can be of great help. For more: https://www.evokelearning.ca/reading-remediation/wise-reading-remediation-for-english-students-grades-1-12/