The EEF is an independent charity that operates in England and that was founded with the help of £125 million of UK taxpayer money. Its mission is to help improve educational outcomes, particularly for the children of disadvantaged and low income households. To this end, it publishes a Toolkit that aims to summarise the evidence for different types of educational intervention, as well as conducting its own randomised controlled trials (RCTs) in order to generate new evidence. Latterly, the evidence from its RCTs has become incorporated into the toolkit.
There are some fairly major implications for Australia. Evidence for Learning (E4L) is a social venture that has licensed the EEF Toolkit for use in Australia and I strongly suspect that this will be a central plank of the ‘Gonski 2.0’ proposals for Australian education when they are finally published. So there is a lot at stake.
I am not ideologically opposed to the EEF. I strongly support the idea of conducting RCTs, as well as the aim of summarising evidence in a way that teachers can use. However, I am a critical friend and have had cause to become more critical over time. Some RCTs have struck me as a little pointless or have not demonstrated what has been claimed. I have even suggested an improvement to the design of RCTs so that we may make stronger inferences from them.
I have also been critical of the EEF’s attempts to compute an ‘effect size’ for the different interventions in its Toolkit and express these in ‘months of additional progress’ because it is a complete nonsense. Instead, the summaries should be open and contested qualitative accounts.
And I have been particularly critical of the ‘metacognition and self-regulation’ strand of the toolkit which, as I explained in the recent article that the UK’s Chartered College of Teaching declined to print, appears to be a chimera; a monster stitched together from quite disparate things. What does a writing intervention where students are explicitly taught how to plan, draft, edit and revise their writing – also known as ‘teaching English’ as Joe Nutt drily observed on Twitter – have in common with ‘philosophy’ lessons where children discuss whether it is okay to hit a teddy bear? Not much. It’s astonishing that such wildly different interventions would be classed as the same thing. Medical researchers, on whose model of meta-analysis the EEF has built, would never group together such diverse approaches, with such different aims, methods and outcomes, and try to compute an effect size as if they are all the same thing.
I have never had a response to any of these criticisms from the EEF. They don’t owe me one but I do think that a body in receipt of public funds should be open to scrutiny and debate. When the Chartered College announced that the next issue of their Impact magazine would focus on ‘developing effective learners’ and would be edited by Dr Jonathan Sharples of the EEF, I speculated that it would focus on metacognition and self-regulation. Sharples replied that I should ‘submit an abstract’ and this struck me as a potential way of initiating the debate that I was seeking.
When the Chartered College indicated that they would not be printing my article, I offered them a suggestion: Why not print it alongside a rebuttal? It seems that Dylan Wiliam was thinking along the same lines.
I am not afraid of a debate. Who would be? Well, perhaps the EEF are.