Education Endowment Foundation discovers small placebo effect (but not for maths)Posted: July 8, 2017 Embed from Getty Images
The Education Endowment Foundation (EEF) have released a report into a three-year randomised controlled trial. Schools were allocated to a control or intervention group and the intervention teachers were provided with the a mentor, time off timetable for training, a number of books to read (I’m sure the authors were pleased about this) and, crucially, a video camera and microphone to record and then review their lessons. The control group received none of these things and carried on as usual.
The result showed that, on a standardised assessment, the intervention group slightly outperformed the control in English (mean of 13.76 vs 13.16, p=0.05, d=0.15) and science (mean of 26.67 vs 26.29, p=0.04, d=0.12) but not in maths where the result was not statistically significant. Note that the English and Science effect sizes were very small.
The most likely explanation for these results is an expectation effect such as the placebo effect. The teachers in the intervention expected it to have an impact and undoubtedly reflected more on their classroom teaching than they otherwise would have done – imagine viewing all those videos. They probably planned a little more rigorously and were given the time and space to bounce ideas off colleagues.
The students in the intervention would also have been aware that something was a little different, particularly given the dirty great video camera at the back of the room. And this might have affected their expectations and behaviours.
Funnily enough, this is not the inference that we are invited to draw. The books and training were all based on something called ‘dialogic teaching’ where teachers ask ‘higher order’ questions, based on the work of Vygotsky and Bruner and all that kind of thing.
So we are meant to conclude that it is the dialogue teaching that is effective. This sits well with the narrative that the EEF have been building around ‘meta-cognition and self-regulation‘ strategies and has already been taken up in breathless news reports (hat tip to @BarryGarelick) as a means to bash traditional forms of teaching.
To me, this demonstrates that the EEF is badly in need of reform. They may have already jumped the shark with Philosophy for Children but there is still a chance to get things back on track. We can’t really eliminate expectation effects in studies like this, but the use of ABC designs, where one intervention is pitted against another, would be a step in the right direction.
Correction: The result for English is not technically statistically significant because it has the value of p=0.051 to three decimal places.