The Reading Recovery moral dilemma

Embed from Getty Images

There has been a debate raging in my Twitter notifications between Dylan Wiliam and, from my perspective at least, everyone else. Wiliam’s position, as I understand it, is that Reading Recovery is an effective intervention for struggling readers that, perhaps more importantly, has been shown to work at scale. Although Wiliam thinks it is based on flawed science, it is better that struggling readers are given access to an intervention that is available now than given nothing at all. Although systematic synthetic phonics is a promising approach that aligns with cognitive science, there is no reason to think it will be quick or easy to either change the way reading is taught in schools or develop and deliver systematic synthetic phonics interventions to remedy poor reading instruction. Why let the best be the enemy of the good?

This seems like a reasonable, pragmatic argument. So why has it attracted criticism from so many people?

What if Reading Recovery really does work?

If Reading Recovery is an effective intervention then we are placed into a moral dilemma.

One of the interesting aspects of education research is its indifference to mechanisms of action. For instance, I have been a critic of the Education Endowment Foundation’s pursuit of Philosophy for Children. I have criticised the way the initial study was analysed, but there is a more fundamental question: How does having debates about whether it is OK to hit a teddy bear improve reading performance, as is claimed? When I ask this question, I do not get an answer, not because I have somehow stumped the researchers but because, to them, it is not even a question. Mechanisms don’t matter.

You see this indifference all across the domain of education research where interventions are tested. It is unscientific for two reasons. First of all, scientists should want to understand how the world works and indifference to mechanisms is the opposite of this. Secondly, it closes down scrutiny by other researchers. If researchers specify a sequence of cause and effect then we give other researchers the option of testing links in that chain in different ways. If they do not, they may dismiss any research that does not exactly replicate the research that was initially done.

In the case of Reading Recovery, I think mechanisms matter a great deal. I don’t think that using multi-cuing strategies is likely to help struggling readers – if anything, it is likely to cause harm by driving a reliance on word-guessing that breaks down for more complex texts – and so I would be surprised if the Reading Recovery techniques themselves have an effect. Instead, I suspect any effect of Reading Recovery is due to a combination of a placebo effect, additional reading practice and one-to-one tuition. If so, we could test each of these separately. If one-to-one tuition is the key then we can ditch the expensive Reading Recovery training and just implement one-to-one sessions. This would have the advantage of avoiding issues such as reinforcing multi-cuing and it is something that could be done pretty much straight away.

There is also the concern that the ‘good’ in this case is the enemy of the ‘best’. It seems to be the experience of many of us that teachers trained in Reading Recovery tend to become the literacy experts in their schools. This then has a backwash effect where strategies such as multi-cuing become the bedrock of initial reading instruction, making it harder for systematic synthetic phonics to make inroads.

What if Reading Recovery does not work?

Although I find it plausible that Reading Recovery has some sort of effect, I am less certain of this than I used to be. New Zealand has been using Reading Recovery for the longest time and there is little evidence of a positive effect. I recently looked at a UK study that purported to show significant effects for Reading Recovery but it appeared to have methodological flaws. The initial ‘I3’ trial in the US seemed to have similar problems. These often revolve around the way students are included in the programme. Anecdotally, some schools appear to ignore the requirement to place the students who are struggling the most into the programme on the basis that they think it will be ineffective for those students. Similarly, some students don’t complete the programme, for whatever reason, and are often excluded from the data analysis.

A later I3 trial in the US seems to have avoided some of these problems by selecting students using scores on initial assessments, pairing them up with similar students then assigning one member of each pair to Reading Recovery and the other to a control. However, when you look at missing data, there is a large disparity. 4136 in the RR group have data and 756 have missing data. Conversely 3719 in the control group have data and 1173 have missing data (note they excluded both members of a pair from the analysis if one had missing data). I understand that the appropriate test of statistical significance for data like this is McNemar’s Test, but I don’t think I have all the data I need to do this. Nevertheless, if attrition is caused by students moving schools then we should expect an equal rate of attrition from both groups, so this seems like an odd difference and makes me wonder if something funny has happened here.

So perhaps there is only really one good trial that shows the effectiveness of reading recovery and that trial has strange attrition rates. Weigh that against the epidemiological failure of the programme in New Zealand and New South Wales and it is reasonable to hold the view that the effectiveness of Reading Recovery has not been demonstrated.

How could a programme involving an obvious placebo, one-to-one tuition and extra practice result in no overall effect? Perhaps the negative effects of strategies such as multi-cuing outweigh all of these potentially positive ones.

In that case, there is no moral dilemma at all.

In my next post, I will take a closer look at the problem of implementation


17 thoughts on “The Reading Recovery moral dilemma

  1. Tom Burkard says:

    Our 2009 report––convinced the Coalition to cancel central funding for this egregious beanfeast for failed teachers and the Institute of Education–where Wiliam has long been engaged. One of the ironies of this is that School Minister Nick Gibb, who was strongly opposed to RR, worked for KPMG prior to entering politics. The move was broadly welcomed by schools in England, who almost always felt they could put the eye-watering costs to much better use. At that time, there was almost nothing in the way of independent evidence of RR’s efficacy, and KPMG’s evidence was laughable. They have recently been caught up in scandals:

    • Thanks for the links. Personally, I am more interested in the arguments of Wiliam and Gibb – arguments I can only assume express genuinely held opinions – than I am in their past connections.

  2. Luqman Michel says:

    You said,“First of all, scientists should want to understand how the world works…”

    Most scientists and educators who write on ‘dyslexia’ and remediation etc come from countries where they speak only English and are unwilling to listen to those who come from countries that speak a few languages.

    The white folks think they know everything and refuse to listen to anecdotal evidences of others even though they have corroborative evidences from the Western world.

    It took me more than 5 years to get the Western world to accept the fact that Phonological awareness deficit cannot be the cause of kids being unable to read – definitely 20% of kids around the world cannot be having this problem. Definitely all my past students who could read in Malay but not in English could not have had phonological awareness deficit.

    • I don’t think many people do suggest that a phonological awareness deficit causes kids being unable to read. That’s not what this debate is about, as far as I am aware. The main argument is that it is a lack of phonics instruction that causes reading difficulties.

      Please do not ascribe motivations or attitudes to the colour of peoples’ skin. If you do so again, I will block your comments.

      • Luqman Michel says:

        Sorry about that remark I made.
        You said, “I don’t think many people do suggest that a phonological awareness deficit causes kids being unable to read. ” This was what was all over the internet from the time I took in my first student in 2004 until around 2015.

        I wrote to Prof. James Chapman of Massey University in 2010 and then to Dr.Joe Torgesen who were adamant about kids being unable to read because of phonological awareness deficit.

        I mentioned the above as that was what was said to be the cause of kids being unable to read.

        You said “The main argument is that it is a lack of phonics instruction that causes reading difficulties.”
        If you do check the times when ‘Whole language’ was taught and when ‘Phonics’ was taught there were still many kids leaving school as illiterates. During both times many kids were also able to read well. Most of the teachers now in Australia and NZ are from the Whole Language period.

        I find that it is the teaching of wrong sounds of alphabets that is the main cause of kids predisposed to shutting down being unable to read. It is not the lack of phonics instruction.

        Many kids who are taught sounds of alphabets wrong reject what is subsequently taught correctly. These are the kids whom we should study. These are the kids who leave school as illiterates.

        We should ask as to how is a short period of remediation able to get them to grade level and maintain them at grade level.

  3. Luqman Michel says:

    You also said,“…they may dismiss any research that does not exactly replicate the research that was initially done.”

    My research over 15 years from all my students reveal that kids predisposed to shutting down disengage from learning to read because of confusion from having been taught the alphabet sounds wrong. (This research has yet to be done by others.)

    The teaching of wrong sounds of alphabets is now perpetuated by BabyTv from the UK which is airing the following programme in more than 100 countries. One of the committee members of Reading Reform Foundation of UK is unable to express an opinion on whether the following video clip is teaching the wrong sounds of alphabets. If a committee member of RFF of the UK is unable to express an opinion then something is really wrong.

    I have written to another member who is representing RFF and she has promised to respond in the new year.
    Here is the video clip that I am trying to get removed from being aired on TV across the world.

  4. Luqman Michel says:

    You said, “….making it harder for systematic synthetic phonics to make inroads.
    New Zealand has been using Reading Recovery for the longest time and there is little evidence of a positive effect.”

    How can synthetic phonics make an inroad when teachers are teaching the wrong sounds of alphabets?
    It is a fact that when remediation is taught with proper sounds of alphabets a majority of the kids classified as dyslexic who are in fact shut down kids are able to read. I bet that when synthetic phonics is taught properly the kids needing remediation will reduce drastically.

    If the teachers ask kids who come for remediation the sounds of alphabets I bet that almost all will sound out consonants with extraneous sounds.

    The above is an easy exercise to carry out by any teacher who teachers remediation classes..

  5. AB says:

    Thank you for putting into words a nagging feeling that’s been lurking in my mind for years : that education research, and teachers, do not seem concerned with the mechanisms of action. There would be a lot of power in understanding such mechanisms – we could generalise and manipulate them to fit other problems. For instance, I have been interested in the idea that – when weak students highlight key words and try to guess what a diagram is asking them to do, they are following a strategy akin to multi-cuing. The mechanisms of action for intervention and dealing with misconceptions would be very useful but are sadly under-represented in any CPD I’ve been to. If we understood the processes at work, we could then bring to bear with the knowledge of their strengths and limitations – or not if we realise they would be inappropriate. It would solve a few issues.

  6. Walt says:

    Think mechanisms have been covered better with recent process evaluations in EEF reports. It’s not so much the mechanism but the desire to use standardised measures that sometimes causes a disconnect between the content of an intervention and the measure.

  7. Gay Parsons says:

    One odd thing about RR was that they moved them out of the program if they weren’t making enough progress then fortunately they gave them to me using synthetic phonics and decodable texts but it’s so hard to remediate the students from the RR program, to undo the guessing of words and using the picture and the meaning, the Speech Pathologist at my school has the same problem.

    • Luqman Michel says:

      Yes, I find it difficult to undo my students guessing of words by using pictures and meaning too. It takes about one month (I teach a one hour session three times a week) to undo guessing. This, I believe is because most of these kids are smart kids who have learned to cope with being unable to read by memorising and then pretending to read. They use pictures in books to help with this. They are unable to read because they have been taught sounds of alphabets wrong. Once they have learned the correct sounds of alphabets they begin to read without guessing or using pictures. Of course there will be words in English where they have to figure out the word and this they will figure out when the time comes for figuring out such words as in the following sentence. ‘I drove down a WINDY road.’
      I believe this ‘Reading wars’ will end if phonics is taught correctly on the onset.
      For kids predisposed to shutting down, initial input, as pointed out by Thorndike in 1913, is paramount.
      Teach kids the sounds of alphabets correctly and I bet kids requiring remediation will be reduced drastically.
      You may like to read testimonials from parents I collected recently on the suggestion of a teacher from Australia.

  8. David Zyngier says:

    Greg you write that “we should expect an equal rate of attrition from both groups, so this seems like an odd difference and makes me wonder if something funny has happened here.” Couldn’t a simple explanation be that children in the smaller attrition group received one on one time with a teacher making them feel wanted and included at the school. That is ” attachment”.

    • My name is Christine Marks. I taught RR for 9 years from 1993-2002. Though I totally regret teaching the scientifically debunked
      MSV cues, I will say I am happy the effective analogy strategy to decode was used…but for only about 5 minutes in a lesson. Each child had their own list of times which was not the same for each child. It was limited, maybe 20 times/phonograms were taught in the whole programmer based on words they knew.
      I am happy because had I not used this researched-based proven strategy in RR, I would not have created the innovative phonics program as a Reading Specialist in St. Paul Public Schools in 2007 and further developing and publishing it in 2012 with great success in many schools with my fellow developer Amy Fink. (www.ladybugliteracy)
      For more of the backstory…my Running Records always seemed to be VISUAL miscues…Duh‼️😕🙄
      Why?? It was obvious to me they needed a decoding/phonics CURRICULUM explicitly and systematically taught!! The meager phonics they got from RR was definitely not going to help them with multisyllabic words they would
      confront in their future reading that they needed to decode without any word they could access in their vocabulary. CLEARLY, a deep phonics program is clearly lacking in RR‼️‼️
      Please take a look at The Chunk Reading Program online and be assured you will see a carefully researched innovative complete phonics program with a LOT of stories using the embedded “chunks” or times/phonograms and plenty of writing to reinforce the analogies! Amy and I have worked very hard over 8 years developing this very effective program and are thrilled to see it being used in many locations: an international high school in Japan, Lincoln International High School in Mpls., a good number of grade schools, public and charter, Two Somali Daycares, Homeschoolers, and over 25 adult literacy sites in South Dakota teaching new immigrants to read. We are very happy to see something SOO good coming out of
      a RR program that taught the SOO WRONG MSV strategy to those strugglers who needed the foundational phonics more than anyone!! Any questions? Check out:
      Thank you for letting me vent AND for allowing me to share The Chunk Reading Program🐞📚‼️
      P.S. We’re presenting at The International TESOL Conference in Denver, Colorado April 3rd! We’re especially excited to present this program to the teachers of ELL students! By learning to quickly decode and become proficient readers with our program, ELL students will read and gain the vocabulary they so sorely need.
      http://www.Ladybugliteracy Go Chunks🐞

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.