Your role in changing education research

Education research can lack relevance. It often takes a postmodern and slightly silly stance, closely related to what the recent ‘Sokal Squared‘ hoaxers refer to as ‘grievance studies’. Not only does such research lack relevance to teachers, there is growing evidence that the whole identity politics project leaves the public at large pretty cold. As an aside, I fear that an adoption of identity politics by mainstream left-of-centre politics could shut them out of power for many years to come. But that is not what this post is about.

There have been a number of attempts to refocus education research on topics and methods that are of more practical significance. Probably the largest of these projects so far are the Education Endowment Foundation in the U.K., Evidence for Learning, its franchise in Australia and similar endeavours in the U.S such as I3. These projects tend to conduct their own large-scale randomised controlled trials and/or summarise the results of other educational trials using meta-meta-analysis.

There are two main problems. Firstly, the large-scale randomised controlled trials that these organisations conduct are often ‘under-powered’. Due to the fact that the test subjects are whole schools, you need a lot of schools involved in order to gain a clear result and many of the trials hit the floor of the number required. Such an environment makes it hard to runs trials with two experimental conditions plus a control – the best way of working out which approaches are the most effective.

Meta-meta-analysis is no substitute, adding only an illusory rigour to the comparison of teaching methods. It may be possible to do this kind of analysis well in the future but we haven’t managed it so far.

Aware of these difficulties, it is tempting to look to cognitive science as a more solid foundation for making inferences about teaching methods, due to the fact that it tends to be more robust. The problem – and it’s not really a problem – is that much of the relevant research tends to be conducted with undergraduate psychology students. But what if you could do some cognitive science research yourself, in your own school and in your own classroom?

You can.

I’ve been conducting research for four years now as part of a PhD course. If you don’t have the prerequisites to start a PhD then you can consider an experimental Masters. In Australia you can even convert your experimental Masters into a PhD over time if all is going well.

What does this look like? I recommend small randomised controlled trials that are randomised at the student level. In other words, you randomly allocate students to one of two or three groups. The key to obtaining worthwhile results is to change only one tiny thing between the groups. This is the opposite of trialing a whole package of measures in the style of the Education Endowment Foundation. For instance, in my research I change the order that two events happen for my two groups of students and attempt to measure the effects on learning. This kind of research helps build the science of learning, piece by verifiable piece.

I recommended small randomised controlled trials be part of the mix when Evidence for Learning was established but, as with everything else about improving the teaching profession, we will probably have to build this ourselves. If you are interested, you need to find a supervisor with expertise in experimental work (if you are such a supervisor then feel free to mention this in the comments) and you need to be aware that everything you do will be subject to rigorous ethics approval processes – you can’t just start running experiments. That’s actually a good thing. And you also get access to research papers.

So please consider getting involved. What’s stopping you?


3 thoughts on “Your role in changing education research

  1. How small is small? Say you have 3 classes, randomise students into them, change one small thing. Unless the same teacher teaches each class, there is enough variability just there to affect results. As I see it, only very large scale studies can get enough statistical power to overcome that problem. Also, even with randomising the students, wouldn’t you still, randomly, get enough different spreads of ability to make results inconclusive?

    • You would need to use the same teacher, in my opinion, although some designs don’t and I believe that’s a flaw. You don’t actually need that many students to overcome the issue of variability. Think of the fact that the EEF often has something like 9 or 10 schools in each condition. If you have three classes randomised at the student level then you can easily beat that.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.