The working title of my book was The Truth about Teaching: An evidence-based guide for new teachers. I changed ‘evidence-based’ to ‘evidence-informed’ prior to publication.
I think we can perhaps agree that education as enacted across the English-speaking world is not generally evidence-based, but the question is whether it could be. Perhaps it is just a case of making teachers aware of the existing evidence? Although I am committed to doing this, I don’t think we actually have anything to inform teachers about that is as solid as the term ‘evidence-based’ implies.
Why is this? Many, perhaps most, education studies are methodologically flawed, with many sources of confound and bias. To counter this, the Education Endowment Foundation was founded in the UK in 2011 to run large-scale randomised controlled trials. However, its insistence on mainly comparing doing something with doing nothing means that you can pretty accurately summarise the totality of the findings: Doing something is usually, but not always, a little better than doing nothing.
Possibly the most useful results so far are negative findings, where approaches that have seemingly delivered miraculous effects elsewhere fail to replicate. A good example of this is Let’s Think Secondary Science which failed to replicate the previously reported effects of Cognitive Acceleration through Science Education or CASE.
A step towards creating more of an evidence base would be three-armed trials where a promising intervention is compared with an alternative promising intervention and a control. We could then legitimately compare different kinds of approaches with each other. Nobody seems keen to do this, which is understandable in a world were something versus nothing generally produces positive results for a researcher’s preferred approach.
There are other sources of evidence about educational practices. Cognitive Load Theory, for instance, is supported by many small randomised controlled trials that have often been replicated in different contexts. You don’t have to accept the more controversial aspects of the theory in order to recognise this body of evidence. However, it would be a stretch to argue that applying the findings of many small, relatively short duration trials to the design of teaching programmes lasting many months is something that is evidence-based rather than evidence-informed.
And there’s plenty of evidence from correlational studies such as the process-product research of the 1960s. These produced highly suggestive findings about effective teaching, but it would have been even better if these studies had been followed by a series of rigorous experiments that sought to isolate the different factors and test the boundaries in terms of student age and experience.
In my view, a growing problem is the belief in the hocus pocus of meta-analysis (and meta-meta-analysis), where effect sizes from totally different types of studies are compared in the manner promoted by John Hattie or the Education Endowment Foundation’s toolkit. Who needs to go to the bother of running rigorous, well-designed trials when you can take the results of a multitude of bad studies and launder them through the process of calculating a standardised effect size? We can all understand the appeal and perhaps that’s why this approach persists in the face of criticism.
Interestingly, critics of the concept of evidence-based education tend to see randomised controlled trials and meta-analysis as all part of the same deal. We are either unthinkingly buy the whole package or we don’t. We are either in favour of anything involving numbers, however valid, or we have to take the sociological turn towards flatulent, verbose non-science.
This cannot be the way forward for our profession. As teachers, we need to be able to argue about and evaluate the different sources of evidence that exist. I am well aware that partly due to the bad ideas that permeate education, this may sound like a call for teachers to do the impossible. When am I going to find the time to learn about research methods, you may well ask? Well, my answer is that this is not each teacher’s individual responsibility. It is our responsibility as a profession, if we are ever to be a true profession. We need to build structures that are led by teachers, that talk to teachers and circumvent the bad guys who want to tell us what to do and think.
We can become evidence-informed teachers. We can do this.
Filling the pail 
Education, like all subjects involving human behaviour, is so complex that it is all but impossible to identify, let alone isolate, all the factors that could influence the outcome of research. Perhaps the most significant one being teachers themselves–they all come loaded with ideas about the best way to teach. You can’t erase that–even a slight note of disdain can convey the message to pupils that ‘I don’t really expect that you’ll get this”.
So for better or worse, I don’t really think we’re going to get more effective recommendations for teachers than observing what works in real, existing classrooms. I might have minor quibbles with Rosenshine’s 10 principles, but they’re certainly a hell of a lot better than most of the interventions dreamed up in education schools. This isn’t to say that we shouldn’t be conducting a broad range of research, but rather that we should be very careful about interpreting it.
We should also bear in mind that if we’d left it to the research community, synthetic phonics might never have developed. The broad outlines that now inform the most effective pedagogies were developed by teachers working against the overwhelming consensus of the times–Bullock (1975) and the National Literacy Strategy (1998). I worked with two of the more significant pioneers, Sue Lloyd and Ruth Miskin. Both of them freely admitted that they knew relatively little about relevant research findings, but without their work, Clackmannanshire might never have happened.
Pingback: Evidence-informed teaching — Filling the pail | Desde mi Salón
The other thing to be concerned with, and it is hinted in the McKnight & Morgan article, is what some people are calling evidence nowadays. The problem with ‘evidence-informed teaching’ is that it appears to be warping in definition and in some circles literally anything, including gut feelings and conversations, can be called ‘small data’ and used to make judgments on students, activities and methods.
Pingback: Education Is Not Medicine – update – Pocket Quintilian
Pingback: What are trainee teachers being taught about reading? – Filling the pail
Reblogged this on The Echo Chamber.