The working title of my book was The Truth about Teaching: An evidence-based guide for new teachers. I changed ‘evidence-based’ to ‘evidence-informed’ prior to publication.
I think we can perhaps agree that education as enacted across the English-speaking world is not generally evidence-based, but the question is whether it could be. Perhaps it is just a case of making teachers aware of the existing evidence? Although I am committed to doing this, I don’t think we actually have anything to inform teachers about that is as solid as the term ‘evidence-based’ implies.
Why is this? Many, perhaps most, education studies are methodologically flawed, with many sources of confound and bias. To counter this, the Education Endowment Foundation was founded in the UK in 2011 to run large-scale randomised controlled trials. However, its insistence on mainly comparing doing something with doing nothing means that you can pretty accurately summarise the totality of the findings: Doing something is usually, but not always, a little better than doing nothing.
Possibly the most useful results so far are negative findings, where approaches that have seemingly delivered miraculous effects elsewhere fail to replicate. A good example of this is Let’s Think Secondary Science which failed to replicate the previously reported effects of Cognitive Acceleration through Science Education or CASE.
A step towards creating more of an evidence base would be three-armed trials where a promising intervention is compared with an alternative promising intervention and a control. We could then legitimately compare different kinds of approaches with each other. Nobody seems keen to do this, which is understandable in a world were something versus nothing generally produces positive results for a researcher’s preferred approach.
There are other sources of evidence about educational practices. Cognitive Load Theory, for instance, is supported by many small randomised controlled trials that have often been replicated in different contexts. You don’t have to accept the more controversial aspects of the theory in order to recognise this body of evidence. However, it would be a stretch to argue that applying the findings of many small, relatively short duration trials to the design of teaching programmes lasting many months is something that is evidence-based rather than evidence-informed.
And there’s plenty of evidence from correlational studies such as the process-product research of the 1960s. These produced highly suggestive findings about effective teaching, but it would have been even better if these studies had been followed by a series of rigorous experiments that sought to isolate the different factors and test the boundaries in terms of student age and experience.
In my view, a growing problem is the belief in the hocus pocus of meta-analysis (and meta-meta-analysis), where effect sizes from totally different types of studies are compared in the manner promoted by John Hattie or the Education Endowment Foundation’s toolkit. Who needs to go to the bother of running rigorous, well-designed trials when you can take the results of a multitude of bad studies and launder them through the process of calculating a standardised effect size? We can all understand the appeal and perhaps that’s why this approach persists in the face of criticism.
Interestingly, critics of the concept of evidence-based education tend to see randomised controlled trials and meta-analysis as all part of the same deal. We are either unthinkingly buy the whole package or we don’t. We are either in favour of anything involving numbers, however valid, or we have to take the sociological turn towards flatulent, verbose non-science.
This cannot be the way forward for our profession. As teachers, we need to be able to argue about and evaluate the different sources of evidence that exist. I am well aware that partly due to the bad ideas that permeate education, this may sound like a call for teachers to do the impossible. When am I going to find the time to learn about research methods, you may well ask? Well, my answer is that this is not each teacher’s individual responsibility. It is our responsibility as a profession, if we are ever to be a true profession. We need to build structures that are led by teachers, that talk to teachers and circumvent the bad guys who want to tell us what to do and think.
We can become evidence-informed teachers. We can do this.