Is the Education Endowment Foundation chicken?

The EEF is an independent charity that operates in England and that was founded with the help of £125 million of UK taxpayer money. Its mission is to help improve educational outcomes, particularly for the children of disadvantaged and low income households. To this end, it publishes a Toolkit that aims to summarise the evidence for different types of educational intervention, as well as conducting its own randomised controlled trials (RCTs) in order to generate new evidence. Latterly, the evidence from its RCTs has become incorporated into the toolkit.

There are some fairly major implications for Australia. Evidence for Learning (E4L) is a social venture that has licensed the EEF Toolkit for use in Australia and I strongly suspect that this will be a central plank of the ‘Gonski 2.0’ proposals for Australian education when they are finally published. So there is a lot at stake.

I am not ideologically opposed to the EEF. I strongly support the idea of conducting RCTs, as well as the aim of summarising evidence in a way that teachers can use. However, I am a critical friend and have had cause to become more critical over time. Some RCTs have struck me as a little pointless or have not demonstrated what has been claimed. I have even suggested an improvement to the design of RCTs so that we may make stronger inferences from them.

I have also been critical of the EEF’s attempts to compute an ‘effect size’ for the different interventions in its Toolkit and express these in ‘months of additional progress’ because it is a complete nonsense. Instead, the summaries should be open and contested qualitative accounts.

And I have been particularly critical of the ‘metacognition and self-regulation’ strand of the toolkit which, as I explained in the recent article that the UK’s Chartered College of Teaching declined to print, appears to be a chimera; a monster stitched together from quite disparate things. What does a writing intervention where students are explicitly taught how to plan, draft, edit and revise their writing – also known as ‘teaching English’ as Joe Nutt drily observed on Twitter – have in common with ‘philosophy’ lessons where children discuss whether it is okay to hit a teddy bear? Not much. It’s astonishing that such wildly different interventions would be classed as the same thing. Medical researchers, on whose model of meta-analysis the EEF has built, would never group together such diverse approaches, with such different aims, methods and outcomes, and try to compute an effect size as if they are all the same thing.

I have never had a response to any of these criticisms from the EEF. They don’t owe me one but I do think that a body in receipt of public funds should be open to scrutiny and debate. When the Chartered College announced that the next issue of their Impact magazine would focus on ‘developing effective learners’ and would be edited by Dr Jonathan Sharples of the EEF, I speculated that it would focus on metacognition and self-regulation. Sharples replied that I should ‘submit an abstract’ and this struck me as a potential way of initiating the debate that I was seeking.

When the Chartered College indicated that they would not be printing my article, I offered them a suggestion: Why not print it alongside a rebuttal? It seems that Dylan Wiliam was thinking along the same lines.

I am not afraid of a debate. Who would be? Well, perhaps the EEF are.


5 thoughts on “Is the Education Endowment Foundation chicken?

  1. Had not realised the editor of the Impact Magazine issue you were submitting to was from EEF. No more explanation or conjecture required as to why your article was rejected then.

  2. Pingback: Accuracy, opinions and tone – Filling the pail

  3. Given the impact that this type of research has on educational policy and the daily life of students and teachers, there has to be scrutiny and open debate. But the lack of open debate in publications seems to be a wide-ranging problem. Prof Scott Eacott had a similar problem,
    ‘Disappointed that SLAM declined my offer to write a response to Hattie’s reply to my paper. Dialogue & debate is not encouraged/supported.’

    In your recent blog about John Sweller’s theory of cognitive load. Sweller said,
    ‘But as a frequent reviewer, I learned long ago that my reviews are just as incompetent as the worst. We all try our best but we are attempting to judge work that we are clearly inept to judge…

    I have at times, advanced suggestions that many felt were outrageous. Some of those suggestions now seem to be
    considered self-evident by many in the field. How did I change people’s views? I do not think I did. Rather, people retired or died to be replaced by younger people who did not have to carry the burden of their own long history.’

    So as you said Greg, ‘mere bloggers of the world unite’

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.