Is Jeffrey Bowers right that there is no evidence to support phonics teaching?

I think it is uncontroversial to claim the general consensus among reading researchers is that phonics teaching is a critical component of early reading instruction. I have been reading a paper by Jeffrey Bowers and I think he would agree this is the consensus while disputing the evidence in support of it.

Bowers’ argument is detailed and, at times, arcane. I am not a reading researcher. Nevertheless, I will need to dig into some of the weeds to try to both understand and evaluate Bowers’ argument. This overlong blogpost will therefore not be for everyone.

Where are we coming from?

I am a researcher working in the field of cognitive load theory. I am also a teacher, a parent and a blogger with a lot of experience of ideological resistance to phonics teaching and some experience of how reading is taught in the wild. All of these incline me towards the systematic teaching of phonics. I am aware that Bowers’ paper will be used by phonics sceptics to bolster their argument and that predisposes me to find fault in it. Bear that in mind.

Bowers is a professor of psychology from the university of Bristol in the UK. He has written about neuroscience and its potential to be applied to education. He also has an interest in a method of reading instruction known as ‘structured word inquiry‘ which is promoted by his brother, Pete Bowers. This is not a secret and Jeffrey Bowers has been quite upfront about this link. I think it is fair to claim that Bowers’ contention is the evidence that phonics is superior to other forms of reading instruction disappears under close examination and we should therefore look to see if other methods, such as structured word inquiry, are a better bet. I am happy to be corrected on that.

What is phonics?

To determine whether phonics teaching is effective, it is necessary to understand what we mean by this. Unfortunately, this sends us into one of those rather tedious discussions of definitions that I generally try to avoid.

In the Bowers paper, he identifies three broad approaches: no phonics, unsystematic phonics and systematic phonics. A critical point of the paper is that, in his view, there is no evidence for the superiority of systematic phonics over unsystematic phonics and default classroom teaching as well as approaches such as ‘whole language’ can be characterised as involving unsystematic phonics.

Based upon my experience and a reading of the paper, I think we need to be more explicit than this. To my mind, phonics is a body of knowledge about the relationships between letters or groups of letters (graphemes) and the sounds they represent (phonemes). In the literature these are known as grapheme-phoneme-correspondences or GPCs. For instance, the grapheme ‘ch’ can be used to represent a number of different sounds such as in the words, ‘cheese,’ ‘chivalry’ and ‘chiropractor’.

Phonics teaching therefore becomes the teaching of these GPCs. Systematic phonics teaching is therefore a planned and sequenced approach to teaching these GPCs that follows some kind of logical development.

Some like to point out that, due to the way that many different languages have influenced English, there are a wide range of GPCs which are often overlapping or redundant. In other words, English has a ‘complex orthography’. However, far from pointing to the futility of teaching GPCs, this leads to two conclusions. Firstly, if a reader can narrow down the sound in a word to one of three then he or she can try out all three and see which fits a word known from oral vocabulary. So that’s still helpful. Secondly, a sophisticated approach to phonics may also teach the general rules that govern when and where different graphemes correspond to particular phonemes by, for instance, considering the source language or morphology. Morphology is the study of morphemes, the smallest units of meaning within a word such as ‘ing’ or ‘ed’, and it can be particularly useful when selecting which graphemes to use when spelling words.

Structured word inquiry apparently focuses on teaching morphology from the start of reading instruction.

A key issue in this discussion is the nature of unsystematic phonics instruction. For example, imagine a teacher who asked students first to predict a word from context or maybe guess it from a picture cue. As a last resort, the teacher might ask the student to consider the first letter of the word and what sound this may represent. Is this unsystematic phonics? Teaching GPCs this way would obviously be a very long process because, as a method of last resort, the rate at which students will encounter GPCs will be low. It is also likely that many GPCs will be omitted, either because the are part of sight words or because they don’t tend to occur at the start of words (e.g. ‘ck’) or the position of a particular grapheme in a word affects the related phoneme. In this instance, can we claim that phonics is being taught unsystematically? I would suggest a better description would be that a partial coverage of phonics is being taught unsystematically.

[Incidentally, the fact that the position of GPCs in a word effects the sounds they represent is a reason why the whole language trope that you could spell ‘fish’ as ‘ghoti’ is false. The ‘gh’ grapheme would never represent the ‘f’ sound in ‘fish’ when placed at the start of a word.] 

From the start

For Bowers, part of the issue with the scientific consensus on phonics is its focus on systematically teaching phonics (which I interpret to mean teaching GPCs) ‘from the start’:

“There is a widespread consensus in the research community that early reading instruction in English should emphasize systematic phonics. That is, initial reading instruction should explicitly and systematically teach letter (grapheme) to sound (phoneme) correspondences. This contrasts with the main alternative method called whole language in which children are encouraged to focus on the meanings of words embedded in meaningful text, and where letter-sound correspondences are only taught incidentally when needed (Moats 2000).”

Despite the fact that Bowers can supply ‘countless quotes’ agreeing with this consensus position, he finds no evidence to support it. This is an odd position which I am going to critique with what initially may appear to be an unfair argument based upon structured word inquiry, but that I hope will illustrate a key issue.

Peter Bowers is keen to stress that structured word inquiry also teaches GPCs ‘from the start’ and he has produced a video to demonstrate this. Why is structured word inquiry, favoured by both Bowers brothers, teaching GPCs from the start given Bowers’ claim about the evidence in support of this? Well, as ever, the debate seems to depend on putting an awful lot of weight on the difference between ‘systematic’ and ‘unsystematic’.

Indeed, the Peter Bowers video does hint at an unstructured way of teaching GPCs, but I am dubious that this can be described as ‘from the start’.

The example he gives is the sentence, “Mom says she wants a cat.” Apparently, the child can read all the words apart from ‘cat’. How can this be, if this is teaching reading ‘from the start’? Either the child has already been taught all of the relevant GPCs or they have memorised sight words (my eldest daughter was given a set of ‘golden words’ to work on memorising each night) or they are perhaps using a predictable book where every page is a variant on, “Mom says she wants a…”. If the child has learnt sight words then I would challenge whether GPCs are being taught ‘from the start’. If the child is responding to a predictable book then I would challenge whether they can actually ‘read’ the words ‘Mom says she wants a’. What exactly is in dispute here? That there is no reason to teach GPCs in a logical order? What is the reason not to?

The logic of systematically teaching GPCs – planning which ones to teach and in which order – is that you can start with the ones that give you the biggest bang for your buck and get the child reading meaningfully as quickly as possible. Which leads to the next point.

Reading for meaning

Bowers perpetuates the idea that alternatives to systematic phonics are ‘meaning-based’ as if phonics somehow is not and instead children are learning how to read aloud statements of the first and second laws of thermodynamics without any idea of what they mean.

Many phonics advocates subscribe to the ‘simple view of reading‘. Again, we could debate definitions, but briefly stated, this contends that reading ability depends on two factors, oral comprehension and decoding. Oral comprehension is our ability to comprehend spoken text. Decoding is the ability to turn squiggles on the page into something equivalent to spoken text (I have been slightly ambiguous here because there are debates about exactly how this happens). If you can decode the words on a page, you could read them out aloud accurately, if required, although you may not necessarily know what they mean. Overall reading comprehension is therefore the product of these two abilities. If you have zero decoding ability or zero oral comprehension, you are not going anywhere (more on this later).

Early phonics teaching does not have to focus on teaching the meaning of words because most of the words used will be deliberately chosen to be within the child’s oral comprehension ability. Instead, children need to gain decoding ability in order to unlock these words. Few early readers will need to have the meaning of, “Pip snapped a stick,” explained to them. Maybe some will have very low oral comprehension or might not have come across a word like ‘snapped’. Maybe some won’t realise that the unfamiliar word ‘Pip’ is a name. But, by and large, instruction in meaning is unnecessary. This does not suggest that meaning is unimportant. It is a critical consideration – zero oral comprehension means zero reading comprehension.

On a personal level, I would add that, in my experience, the ‘aha’ moment that goes with unlocking meaning from text in this way is highly motivating for young children.

Meta-analyses

Bowers surveys the meta-analyses of reading research relied upon by various government panels and researchers to support their claim that systematic phonics is effective. After reanalysing them, he takes the view that although there is evidence that phonics is better than no phonics, there is no evidence that systematic phonics is superior to unsystematic phonics. He also claims that the effects of phonics interventions wash-out over time.

I have some sympathy for Bowers’ complaints about the various meta-analyses. Both the meta-analyses, and Bowers’ critique, rely on comparing effect sizes for different experimental versus control groups, but I am not convinced that an effect size is as stable a metric as many assume. Yes, at least in this case they all relate to the same thing – reading instruction – and not a whole menagerie of different outcomes as in, for example, the Education Endowment Foundation’s meta-meta-analysis of ‘metacognition and self-regulated learning‘, but they still apply to interventions of different durations, sometimes with whole cohorts and sometimes as interventions with specific cohorts, sometimes as initial instruction and sometimes as later catch-up interventions, sometime with non-native speakers and so on.

As the actual studies are embedded in individual papers which are then nested in the Bowers paper, I cannot easily see what the ‘unsystematic phonics’ control conditions are. This is critical to the argument because I doubt they resemble whole language instruction, as Bowers claims. I am sure whole language can be taught with great attention to GPCs and perhaps this is even the intention, but the rhetoric about ‘barking at print’ and so on tends to point in the other direction.

For instance, Brian Cambourne is a notable Australian proponent of whole language instruction and he advocates teaching GPCs only through writing. Is this what we mean by unsystematic phonics? It certainly seems far less focused on GPCs than, say, the structured word inquiry approach. Moreover, Cambourne incorrectly claims that phonics advocates believe ‘reading is decoding’ – phonics advocates  do not believe this if they adopt the simple view of reading – and describes this as ‘read-i-cide’. It is clear what message teachers are supposed to take from this.

As I have not accessed all of the original papers behind these meta-analyses, I cannot accurately assess Bowers’ claims about them. He may be right. However, as I read his paper, there does seem to be an element of convenience about what he chooses to exclude as he reanalyses the data. Sometimes a large effect size is excluded. At other times, there is a lengthy description of the relative outcomes for different groups of students such as non-native English speakers and so on.

What we have to bear in mind is that this is all post-hoc. Bowers already knows all of the findings and is reanalysing them with a specific hypothesis in mind. That is something very different to starting out from scratch and conducting a study or a new meta-analysis. This is why we have seen a wider trend towards preregistered trials with pre-defined outcome measures.

For those of us who cannot analyse the data ourselves, we are left with weighing the conclusions of all of the teams of researchers who conducted these original analyses prospectively against the post-hoc reanalysis of Bowers. Even if we accept his reinterpretation as correct, we are accepting that phonics is superior to no phonics and are left with this claim that unsystematic phonics – whatever that is – is as good as systematic phonics. It is as if research has discovered that kids with desks in their rooms do better on exams than those without, but there is no statistically significant difference between those who have messy or tidy desks.

Bowers also makes claims that the effects of phonics wash out over time. This is hardly surprising and is a result of basic logic. Presumably, post intervention, most kids will go back to getting the default reading instruction diet. Clearly, the effect of X versus Y is always going to be greater than the effect of XYYYYY versus YYYYYY or XZZZZZ versus YZZZZZ. Think of The Princess and The Pea.

Public Policy in England

Finally, Bowers examines public policy in England. Phonics was mandated in 2007 following the 2006 Rose report and a phonics check was introduced in 2012 to ensure all schools were following this mandate. Surely, Bowers suggests, we should see a signature of that in subsequent standardised reading tests?

No, I am not convinced that we should. Standardised reading tests assess reading comprehension. This is the product of oral comprehension and decoding. Phonics only acts on decoding. If we do not also improve students’ oral comprehension – their knowledge ‘of words and the world‘, as E. D. Hirsch puts it – we should not necessarily see an improvement in reading comprehension (which is why advocate for a knowledge-rich curriculum).

Bowers makes a number of similar arguments and so I shall focus on just two because I think they illustrate the no-win position that he places phonics in.

The Key Stage Two standardised assessments changed in 2016 in order to make them more rigorous. The first cohort of students who completed the phonics check in 2012 would have sat these assessments in 2017. Bowers points to similar scores in the 2016 and 2017 and 2018 assessments and suggests this is evidence that phonics had no effect.

However, phonics has been mandated in England since 2007. Yes, 2012 added another accountability layer but it was well heralded and so a gradual move towards more phonics teaching seems more likely than a sudden step-change in 2012. Anecdotally, I understand that some schools reacted to the check by asking students to memorise nonsense words and that some schools are still, in 2020, using the three-cuing system criticised by Sir Jim Rose in 2006. So I find it hard to assume we went from zero to systematic phonics in 2012. I am also unsure as to how the UK government standardise the new Key Stage Two assessment and whether it would accurately pick up improvements in reading.

A related question then arises about the effect of mandation in 2007. Following Bowers’ previous logic, this should have affected performance on the old version of the Key Stage Two assessments. The first cohort sitting these assessment would have been in 2012 and the scores do indeed increase. However, this reversal of the previous finding is also dismissed by Bowers because maths and science scores rose at the same time and so this could be grade inflation.

This is, of course, all plausible, particularly to someone who is not expecting to see massive effects on standardised assessments, but it does again seem like we are starting with the conclusion and working back from there. Flat results provide evidence for the hypothesis but rising results also provide evidence for the hypothesis.

Where to from here?

I would like the the Education Endowment Foundation (EEF) to set-up a three-armed trial of a high quality systematic phonics programme, such as Sounds-Write, versus the best available unsystematic (or even ‘analytic’) phonics programme and a business-as-usual control. Structured word inquiry may be able to fulfil the role of the unsystematic phonics programme, although, given the discussion above, I am unsure if it is ready for teaching beginning reading. Such a trial may then allow us to pick some of this apart.

As for Bowers’ argument, it is hard to judge. What he desperately needs is something prospective i.e. to make a prediction before a study has been conducted and see that prediction supported by the subsequent evidence. Given the interest in structured word inquiry, there is obviously an advantage for Bowers if something like the EEF study I propose above could be run that involved structured word inquiry.

Those with the means may want to dig back through the papers that sit behind the meta-analyses that Bowers critiques. Setting aside the various technical criticisms Bowers makes, the crucial issue would be to determine the exact nature of these unsystematic phonics programmes that Bowers’ suggests are as effective as systematic phonics under his analysis. Even if Bowers’ analysis does not hold and the more conventional view prevails, it would still be a useful exercise.

In the meantime, I see no reason to ditch systematic phonics. Even if Bower’s turns out to be correct in all of his critique, phonics clearly works and I don’t think anyone in this debate is arguing that making it systematic is a cause of harm or that there are advantages to being unsystematic. If nothing else, a systematic approach makes sense from a planning perspective. It also aligns well with other sources of evidence from cognitive science.

19 thoughts on “Is Jeffrey Bowers right that there is no evidence to support phonics teaching?

  1. Interested to note that the Anne Castles reading review paper 2018 concluded there was no convincing evidence for systematic phonics over analytic phonics. Time for some quality studies to sort it out, as you suggest.

    1. Castles et al. claim is that there is no evidence for “synthetic” over “analytic” phonics. They do claim that systematic phonics is best practice (both synthetic and analytic phonics are systematic).

  2. This decade, starting with 2020, may just be the period when the Reading Wars might see some resolution. After a half-century of battle! In the United States there is a lot of pent-up anxiety about the state of reading — not the least coming from the advocacy groups demanding an end to the school-to-prison-pipeline — citing statistics of over 70% illiterates in the prison population. There have been many articles in popular magazines and newspapers highlighting reading problems.

    Coincidentally to Greg’s post today, just yesterday a popular US educator site also brings up the topic of the Reading Wars. See Jay P Greene’s Blog https://jaypgreene.com/2020/01/14/whole-leech-uage-instruction/#comment-709056 The guest writer, Greg Forster, decidedly takes a position against one of the two sides (he is for phonics against whole language). The comments will be of interest to readers of Greg Ashman’s blog. (BTW, thanks Greg A for your detailed analysis in your post.)

    Scanning the 25-page paper by Jeffrey Bowers I note his conclusion: “The ‘reading wars’ that pitted systematic phonics against whole language is best characterized as a draw. The conclusion should not be that we should be satisfied with either systematic phonics or whole language, but rather teachers and researchers should consider alternative methods of reading instruction.” Interesting!

  3. On the face of it, the starting assumptions of Structured Word Inquiry, that: (1) “The primary function of English spelling is to represent meaning”; and (2) “The conventions by which English spelling represents meaning are […] well-ordered and reliable” are nonsense. The whole point of an alphabetic system is that it divorces meaning from symbols, and instead associates symbols with a much more restricted range of sounds. With such faulty starting assumptions, anything that follows will be suspect too.

  4. I really appreciate that Greg is challenging the claims I’ve made in my critique of systematic phonics, but his response does not address the fundamental problems regarding the evidence. Greg makes some points about the theoretical motivation of SSP and criticism of SWI that I’ll ignore for now and just restrict my comments to the evidence (or lack of) for systematic phonics.

    There were two major sets of data I considered in my critique, namely, all the meta-analyses and the reading results in England since SSP was mandated since 2007. Let’s take them in turn.

    1) Regarding the meta-analyses Greg has two concerns. First, he is not convinced of the distinction between systematic, non-systematic phonics, and no-phonics, nor of the claim that whole language includes much unsystematic phonics, writing: “I am sure whole language can be taught with great attention to GPCs and perhaps this is even the intention, but the rhetoric about ‘barking at print’ and so on tends to point in the other direction.”

    I should point out that this is this distinction was made by the authors of the NRP (2000) who compared systematic phonics to a control condition composed of a combination of non-systematic phonics and no-phonics. The authors of NPR also noted that non-systematic phonics was standard in whole language as practiced in the US in the 1980s-1990s (Studies in the UK also showed that non-systematic phonics was common in the UK prior to the legal required of SSP). If you take the NRP findings as evidence for SSP, it seems a bit odd to reject their claim that non-systematic phonics was (and often is) standard in US schools (indeed, it seems implausible to me that there was ever a time when children were given no instruction in sounding out words in school settings). But putting this aside, as a point of logic, if you are going to claim that systematic phonics is better than non-systematic phonics it makes no sense to design a meta-analysis that has a control condition composed of non-systematic phonics and no-phonics. But that is exactly what the NRP did. When Camilli et al compared systematics phonics to non-systematic phonics in the studies included in the NRP the effects were much weaker (in one analysis, no longer significant).

    I do think that this distinction between non-systematic phonics and no phonics is important, but what Greg does not note is that all the meta-analyses I discuss after the Camilli et al. continued to compare systematic phonics to a control condition that included both non-systematic phonics and no-phonics (or in one case, only no-phonics), and I did not correct for this design choice in my summaries of these studies. And even with this design that should inflate the effect of systematic phonics the evidence was weak at best (and in almost all cases null for the most important reading measures). So, the poor evidence for SSP from the meta-analyses not hinge on this distinction that Greg is dubious about.

    Another concern about my summary of the meta-analyses is that I’ve excluded studies to suit my purposes. As Greg writes:

    “However, as I read his paper, there does seem to be an element of convenience about what he chooses to exclude as he reanalyses the data. Sometimes a large effect size is excluded. At other times, there is a lengthy description of the relative outcomes for different groups of students such as non-native English speakers and so on.

    What we have to bear in mind is that this is all post-hoc. Bowers already knows all of the findings and is reanalysing them with a specific hypothesis in mind. That is something very different to starting out from scratch and conducting a study or a new meta-analysis. This is why we have seen a wider trend towards preregistered trials with pre-defined outcome measures. For those of us who cannot analyse the data ourselves, we are left with weighing the conclusions of all of the teams of researchers who conducted these original analyses prospectively against the post-hoc reanalysis of Bowers.”

    This criticism is also mistaken. The first thing to note is that the evidence for SSP was weak even when all these studies were included in the analyses (the results of the full analyses are reported in the paper). Also, in some cases the authors of the meta-analyses *themselves* excluded the studies in some of their analyses because they were obviously problematic. I think there were only two instances where I suggested that that studies should have been removed, and I detailed my reasoning. Here are the explanations I gave in the paper.

    In the case of McArthur et al. (2012) meta-analysis I wrote:“But there are reasons to question even these modest conclusions. One notable feature of the word reading accuracy results is that they were largely driven by two studies (Levy and Lysynchuk 1997; Levy et al. 1999)… This is problematic given that the children in the Levy studies were trained on one set of words, and then, reading accuracy was assessed on another set of words that shared either onsets or rhymes with the trained items (e.g., a child might have been trained on the word beak and later be tested on the word peak; the stimuli were not presented in either paper). Accordingly, the large benefits observed in the phonics conditions compared with a nontrained control group only shows that training generalized to highly similar words rather than word reading accuracy more generally (the claim of the meta-analysis). In addition, both Levy et al. studies taught systematic phonics using one-on-one tutoring. Although McArthur et al. reported that group size did not have an overall impact on performance, one-on-one training studies with a tutor showed an average effect size of d = 0.93 (over three studies). Accordingly, the large effect size for word reading accuracy may be more the product of one-on-one training with a tutor rather than any benefits of phonics per se, consistent with the findings of Camilli et al. (2003). In the absence of the two studies by levy and colleagues, there is no evidence from the McArthur et al. (2012) meta-analysis that systematic phonics condition improved word reading accuracy, word reading fluency, reading comprehension, spelling, or nonword reading fluency, leaving only a benefit for nonword reading accuracy. “

    In the case of Suggate (2010) I wrote: “It is also important to note that 10% of the studies included in the meta-analysis were carried out on non-English children. Although the overall difference between non-English (d = 0.61) and English (d = 0.48) studies was reported as nonsignificant, the difference approached significance (p = 0.06). Indeed, the phonics intervention that report- ed the very largest effect size (d = 1.37) was carried out in Hebrew speakers (Aram & Biron, 2004), and this study contributed to the estimate of the phonics effect size in prekindergarten. Accordingly, the small advantage of phonics (the main novel finding in this report) is inflated when applied to English.”

    And in any case, even if Greg wants these studies included in the meta-analyses, the effects for SSP for spelling, reading fluency, reading comprehension are all very weak or null (the only consistent effects are for decoding regular words and nonwords). This despite the improper design that will likely have inflated the effect (combining nonsystematic phonics and no phonics in control condition), evidence for publication bias that would have inflated the effects, the fact that few RCT studies were carried out, and that many additional problematic features of the studies were identified by the authors of the meta-analyses themselves. Together, this does not add up to strong evidence from the meta-analyses. Greg do you agree or disagree that the evidence from the meta-analyses do not provide compelling evidence for SSP?

    2) Regarding the reading results in England since 2007, Greg makes some points I think are mistaken (for another time), but in any case, he does not expect standardized reading results to increase, writing: “Bowers suggests, we should see a signature of that in subsequent standardised reading tests? No, I am not convinced that we should. Standardised reading tests assess reading comprehension. This is the product of oral comprehension and decoding. Phonics only acts on decoding. If we do not also improve students oral comprehension – their knowledge ‘of words and the world‘, as E. D. Hirsch puts it – we should not necessarily see an improvement in reading comprehension (which is why advocate for a knowledge-rich curriculum).”

    So, if I understand, Greg agrees that there is no good evidence that SSP improved standardized reading results, but he argues that this does not mean that SSP has not worked. I actually agree with this – it is possible that SSP has worked and the benefits have not shown up in standardized SAT, PIRLS, and PISA results. I’m no particular fan of these tests. My point is that the SAT, PIRLS, and PISA results should not be used to support SSP (as often done). If you consider the weak/null evidence for SSP from both meta-analyses and analyses of English reading scores on standardized tests post 2007 you end up with little or no basis for claiming that the “science of reading” supports SSP. Perhaps better research in the future will provide strong support for SSP, but for now, this claim is not justified. Am I missing something?

    If you want an easy introduction to the paper you can go to my webpage and read some of my blogposts: https://jeffbowers.blogs.bristol.ac.uk/blog/

    1. The problem with post-hoc reasoning is not the actual reasoning – which I am not in a position to comment on one way or the other – but that it is post-hoc. Repeatedly, you challenge the authors’ views of their own evidence. For instance:

      “Despite their modest conclusions, the authors are still far too positive regarding the benefits of systematic phonics. In part, this is due to the way the authors summarize the findings they do report. But more importantly it is the consequence of ignoring many of key limitations of the meta-analyses discussed above… One problem with this description of the results is that it does not indicate which measures tended to be significant over the meta-analyses.”

      And so on.

      We are invited to believe that you have retrospectively found fault in the conclusions of many researchers working prospectively (well, to the extent you can when performing a meta-analysis). You may be right and all these other people may be wrong, but to an outsider, that doesn’t seem like a great bet.

      The gaping hole in the paper, as far as I am concerned, may be one that is further answered by looking into the original papers, which I intend to do if I ever get the time. You claim there is no evidence to support systematic phonics being superior to unsystematic phonics and you then, critically, equate unsystematic phonics with whole language and default classroom practice. You doubt anyone avoids teaching GPCs completely, and I agree. But the kind of ‘look at the first letter’ practice I have mentioned above is, in my experience, quite common. Does this equal the unsystematic phonics of the studies?

      You provide evidence that default reading instruction is equivalent to unsystematic phonics by quoting from a UK inspectors’ report that does not appear to be available online and the NRP report. Even if the inspection report were available, it would not tell us much, I fear, because inspectors announce their visits in advance and it is doubtful they see standard practice. As for the NRP report, a section you quoted on Twitter, but not in the paper, reads:

      “In the present day, whole language approaches have replaced the whole word method as the alternative to systematic phonics programs. The shift has involved a change from very little letter-sound instruction in 1st grade to a modicum of letter-sounds taught unsystematically.”

      As I said, I will look into the papers if I get the chance and hopefully, if they provide good descriptions of the different methods used, I will independently answer this question. However, the logic of referring to this evidence implies that you think the evidence shows that teaching a ‘modicum’ of letter-sounds in first grade – which is an apt description the first letter approach I outline – is as effective as a systematic phonics program. Is that your contention?

      In terms of the results for England, I think you risk setting a correlation-causation trap for yourself. If, as you seem to do, we accept that a correlation between rising standardised reading scores and the introduction of systematic phonics proves phonics and phonics alone is the cause of this improvement, any future significant rises in standardised reading scores will, by your logic, prove the value of systematic phonics. I suspect scores in PIRLS and PISA will indeed rise, but as a result of the combination of systematic phonics and a more rigorous curriculum that will also lead to likely gains in maths and science. I am unsure about KS1 and KS2 results because I don’t know how robust these are in measuring gains or declines.

      If this happens, you will then be in the position of explaining why these results also do not support systematic phonics and that will be a difficult task.

      1. I think there is a fundamental confusion here. I am not claiming that the evidence supports the conclusion that non-systematic phonics and whole language works as well as systematic phonics. Rather, I’m claiming that there is currently no evidence that systematic phonics is better. The scientific burden is on researchers claiming that systematic phonics is better than alternative methods.

        I’ve gone through every meta-analysis designed to test the efficacy of systematic phonics and found them all fundamentally unconvincing: none of the analyses that claim to find evidence for systematic phonics even tested the claim (they did not compare systematic phonics to non-systematic phonics), they have mischaracterized their own results, they have not carried out the appropriate statistics to support their conclusions, there are flawed studies that go into their conclusions, few RCTs, evidence of publication bias, etc. None of this is to say that SSP does not work. You can run another meta-analysis that addresses these flaws and perhaps you will find evidence for SSP. Or you can run another better and more powerful study and show SSP works. But till then, the current evidence for meta-analyses does not support SSP. There is nothing post-hoc about finding flaws in all meta-analyses used to support SSP,

        With regards to the Year 1 SATs in England, the results from reading, writing, math and science all started increasing prior to the introduction of SSP, and they continued rising at a similar rate afterwards. This straightforwardly provides no evidence of the benefits of SSP. I agree in the future reading and writing might selectively improve, and this would provide evidence that SSP was an improvement compared to what was on offer before. But till then, the SAT (and PIRLS and PISA) results cannot be used to support the implementation of SSP in England.

      2. “There is nothing post-hoc about finding flaws in all meta-analyses used to support SSP,”
        But I am afraid that there is.

        In the olden days, prior to meta-analysis, researchers summarising the evidence for something would decide which papers to include and which to exclude. The problem here is that they already know the results of the relevant studies and so bias would lead them to accept studies that support their own hypothesis and find reasons to exclude or criticise studies that do not.

        Meta-analysis was introduced as a way to partly combat this. Researchers were required to state, upfront, selection criteria, what evidence they would accept and reject and so on. Ideally, this should be published and scrutinised in advance of then conducting the meta-analysis.

        What you have done is take the results of a meta-analysis and then critique aspects of it in various different ways, knowing the results and with a hypothesis in mind. This process is susceptible to just the same bias as that which meta-analysis was designed to prevent. That is why it is a post-hoc analysis as much as you claim it is not. You should perhaps instead consider doing your own meta-analysis from scratch.

        For me, this issue casts doubt on your conclusions that there is no evidence that systematic phonics is superior to unsystematic phonics, despite the original researchers suggesting that there was such evidence.

        However, if we accept your claim, a second problem arises. You claim this means there is no evidence that systematic phonics is superior to whole language. Clearly, this meta-analysis doesn’t demonstrate lots of things eg the existence of life on mars, but that’s not worth stating because nobody would expect it to. The implication that it provides no evidence that systematic phonics is superior to whole language is that this study is relevant to such a question and could have generated such evidence (as the original NRP authors suggest).

        You indeed do think this is relevant because you identify whole language with the unsystematic phonics construct used in the studies. Then you interpret your claim that there is no evidence for the superiority for systematic phonics over unsystematic phonics as being the same as finding there is no evidence for the superiority of systematic phonics over whole language.
        But why should we accept this mapping? It seems that it was Camilli et al in two papers who coded conditions as 0, 1 or 2 depending on whether then include no phonics, unsystematic phonics or systematic phonics. However, the rubric confuses me:

        Am I reading this right, @jeffrey_bowers? It looks a lot like the difference between 'unsystematic' and 'systematic' phonics instruction as coded by Camilli et al. 2003 is about the presence or absence of direct instruction (unsystematic is still somehow 'organized'). pic.twitter.com/7b96ytwkUD
        — Greg Ashman (@greg_ashman) January 16, 2020

        https://platform.twitter.com/widgets.js

        0 could indeed include some phonics and 1 could include organised phonics. I am therefore not convinced that 1 is a better mapping to whole language than 0. If 0 is the better mapping then the Camilli papers, according to your logic, do provide evidence that systematic phonics is superior to whole language.

        The obvious way to test this is to look at a list of the papers Camilli et al. include in their meta-analyses and how they coded each condition. However, I cannot find such a list and, when asked on Twitter, you could not point to one. So we don’t know. If, for instance, things like reading recovery are coded 1 then this would not represent whole language because it is a one-to-one intervention for struggling readers and not a whole class initial reading programme for all children.

        I am afraid that your approach to the standardised test data does rather betray your post-hoc methods. Why should a genuine improvement in reading not coincide with an improvement in maths and/or science? Can things not improve simultaneously? Reading should have a direct effect on science and maybe maths and a beefed-up curriculum could affect all three.

      3. I am afraid I don’t understand your position. I’m not allowed to find flaws in meta-analyses? Do you think that the flaws I have identified are incorrect, or that I’m being post-hoc in pointing out that the design of the meta-analyses did not actually test the hypothesis that systematic phonics is better than WL, or that the design of the meta-analyses did not actually test whether systematic phonics is better than non-systematic phonics, or that the authors of a meta-analysis need to report an interaction to support their conclusion (where they didn’t test for it), or that abstracts and summaries consistently mischaracterize the findings in the result sections, or that a study in Hebrew is not relevant to reading instruction in English, or that a study with an obvious confound an absurd effect size near d = 3 should not be included in their estimate, etc., etc., etc. Again, I’m not claiming that null effect is true, just that there is no evidence to reject the null hypothesis.

        No meta-analysis other than the NRP even compared systematic phonics to WL, but meta-analysis after meta-analysis is used to support the claim that systematic phonics is better than WL. And in the case of the NRP, there is no evidence that the benefits of systematic phonics over WL extended to reading comprehension, spelling, fluency, or extended over time. Indeed, there is no evidence that synthetic systematic phonics (the form required in England) had *any* benefit over WL in the NRP. Is it ad-hoc to point this out? After reading my critique of the meta-analysis, are you still confident that these meta-analyses provide good evidence for systematic phonics?

        There is similarly no evidence in any standardized test that reading has improved in England since 2007. I’m still not sure whether you agree with this conclusion or not. I do agree that reading might have improved, but it is not evidenced in these standardized tests. Or do you think the pattern of results in these tests do indeed support the conclusion that SSP has been effective in England? If you think this, please point to the outcome that you think supports this conclusion.

        I’d be interested in other researchers commenting on this – especially the claim that I’m being post-hoc in critiquing the meta-analyses (and if so, how you should identify flaws without being post-hoc). I think I’ll leave it there for a while, but thanks for engaging,

        jeff

      4. I find it strange that you do not understand the post-hoc criticism. Surely you are familiar with the reason for the adoption of meta-analyses as I explained above?

        If you did your own meta-analysis, stating clearly your criteria, you may even find you include studies not included in the analyses you are criticising, or you may find you want to critique a study in a way you did not set out in your criteria etc. etc. It makes it a higher level of analysis.

        You are, of course, free to critique whatever you like. However, we are left with weighing a number of researchers doing a systematic analysis against a lone researcher doing a post hoc one. That’s why I find it unconvincing.

  5. I would be interested in thoughts around phonics in Second language acquisition, as a teacher in Hong Kong levels of comprehension are no where near that of native countries for multiple reasons. However, we have found especially among young learners explicit teaching of decoding and reading skills increases confidence hugely, and with that confidence boost comes increased interest and involvement.
    We also have the issue of testing here which focuses on comprehension, even at very young ages, although slowly, ever so slowly, there is a move towards skills, rather than knowledge acquisition. This may better demonstrate the effectiveness of learning to read, rather than reading to learn strategies.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.