Education research – the evidence

---

I posted a flashcard I had made about education research on Twitter. My intention was to provoke a discussion with some challenge to the points I had made. I wasn’t quite prepared for just how much engagement the flashcard would receive. I wouldn’t exactly call it viral, but it’s far more than I am used to.

Please feel free to share my flashcard. pic.twitter.com/XvEdodZAQQ

— Greg Ashman (@greg_ashman) November 12, 2020

As I expected, a number of people asked for the evidence to support these points and so this is the blogpost in which I intend to set this out. On reflection, there are a mix of issues requiring different levels of evidence. For instance, it is really up to those asserting that learning styles exist to provide supporting evidence and not up to me to prove they don’t. In contrast, I do hold the burden of proof for assertions such as that achievement boosts motivation. Nevertheless, I will do the best I can to outline the evidence as it stands for each point.

You do not learn something better if you figure it out for yourself

It is the nature of something like a flashcard that statements are pithy. And it is the nature of Twitter to argue about definitions and multiple interpretations. So let me be clear, I do not contend that being taught something really badly is as good as figuring something out for yourself. However, I don’t expect that anyone honestly interpreted it that way. Instead, take it as a statement predicated on all other things being equal i.e. the best teaching versus the best discovery learning.

The evidence for the effectiveness of discovery learning for learning something new is extremely thin and, in my experience, positive results usually come from comparing it to something that everyone agrees is not the most effective form of instruction such as completing worksheets or listening to a non-interactive lecture. If you want a better survey of the whole field then this paper by Richard Mayer lays out the case against ‘pure’ discovery while also making an interesting point about constructivism. There is also, of course, the seminal paper 2006 by Kirschner, Sweller and Clark that prompted three rebuttals, a reply from the authors, a conference and then a book. The Kirschner et al. paper discusses the ineffectiveness of ‘minimal guidance’. There has been much Twitter-style equivocation about that term and so it is also worth reading a version of this argument that the authors make in American Educator which clarifies that they are focusing on the superiority of fully guided instruction.

However, this evidence is not enough. It could be the case that discovery learning is less efficient than explicit teaching or that fewer students learn through this method, but that those who do manage to learn in this way learn the concepts at some deeper level. One study attempted to test this in the context of primary school children learning about controlling variables in science. As expected, fewer successfully learnt through discovery. However, those who did manage to learn through discovery performed no better on a task that asked them to then evaluate science fair posters to see whether variables had been controlled.

We should perhaps not be surprised. The largest education experiment of all time, Project Follow Through, found that Direct Instruction, that researchers labelled a ‘basic skills’ approach, was more effective than a whole range of models at improving outcomes for early elementary education. Importantly, models that emphasised self-esteem or constructivist models that involved elements of discovery, did not perform better than Direct Instruction on measures of problem-solving ability, reading comprehension or, indeed, self-esteem.

I am also sceptical about how the supposed superiority of discovery learning fits with theory. The messy process of figuring something out and potentially attaching new items to the wrong schema would imply a degraded form of learning compared to a structured approach. It is therefore the burden of advocates of discovery learning to start supplying some solid evidence of its benefits.

Learning styles are a myth

Do people express preferences for how they learn? Yes. However, when we try to improve teaching by catering to these learning styles, we find no effect. So that’s the myth. The concept of learning styles seems to be yet another manifestation of the WEIRD cult of individualism and we may be better off teaching to what students have in common, especially given that we only have the resources to teach them in batches of 25-30.

If you give students choice over how they learn, the often choose the least effective method

This is a key finding of Richard Clark when analysing the results of aptitude-treatment interaction (ATI) studies. Low knowledge students who would benefit from a highly structured approach tended to prefer less structured activities where they could keep a ‘low profile’. In contrast, high knowledge students who were ready to practice independent application tended to want to hold on to highly structured explicit teaching. Similarly, Foster, Rawson and Dunlosky found students were reluctant to choose to study worked examples when they would have been beneficial and Singer and Alexander found that students preferred digital to print texts, predicted they would comprehend digital texts better but actually showed a slight superiority in comprehension when using the print texts.

Again, this chimes with teaching experience. We all know students whose main revision strategy is to rewrite or even just reread their notes, when quizzing is more effective (more later) and we all have seen the group poster task where the lowest knowledge student is allocated the task of drawing the title in bubble writing. So, teachers really need to shoulder the responsibility they are paid for and take charge.

Testing is a highly effective way to boost learning

The act of retrieving something from long-term memory appears to boost that retrieval pathway. This is probably the most solid finding in all of educational psychology. Unfortunately, many people mix up frequent, in-class, low-stakes testing with high-stakes tests and examinations which they (wrongly) perceive to be a bad thing.

Karpicke outlines the argument here. Dunlosky make a teacher-friendly version of it here.

The evidence for the effectiveness of whole-class, interactive explicit teaching comes from a wide range of studies

Probably the largest body of evidence supporting whole-class explicit teaching is a body of studies that largely took place in the 1950s-1970s and are known as ‘process-product’ research. Briefly, researchers would observe classes and log various teacher behaviours. They would then look at the gains students made in assessments of their learning. Brophy and Good summarised this research in the 1980s and described the most effective models as ‘active teaching’:

“Students achieve more in classes where they spend most of their time being taught or supervised by their teachers rather than working on their own (or not working at all). These classes include frequent lessons (whole class or small group, depending on grade level and subject matter) in which the teacher presents information and develops concepts through lecture and demonstration, elaborates this information in the feedback given following responses to recitation or discussion questions, prepares the students for follow up seatwork activities by giving instructions and going through practice examples, monitors progress on assignments after releasing the students to work independently, and follows up with appropriate feedback and reteaching when necessary. The teacher carries the content to the students personally rather than depending on the curriculum materials to do so, but conveys information mostly in brief presentations followed by recitation or application opportunities, There is a great deal of teacher talk, but most of it is academic rather than procedural or managerial, and much of it involves asking questions and giving feedback rather than extended lecturing.”

This research is correlational rather than experimental and so we cannot be certain that the behaviours of these teachers cause the increased achievement of students. However, Rosenshine points out that subsequent to these studies, a range of different experimental studies involving a diverse selection of learning objectives have confirmed these findings. After practice testing, this is probably the next most solid finding we have in education.

Broad world knowledge is essential to reading comprehension

Hirsch makes the argument here. Willingham does so here. I won’t rehearse them. Its’ worth pointing out that the experimentally validated simple view of reading posits that reading comprehension is the product of decoding ability – turning the squiggles on the page into words – and oral language competence – understanding what the words mean. The latter is not just about vocabulary, it’s about being able to form a mental model of what is being discussed and that requires sufficient knowledge.

You cannot just Google facts when you need them

This is linked to the previous point. Partly, this is a common sense argument. If you lack knowledge, how will you know what to look for? How will you know what it is that you don’t know? If you find something and lack the world knowledge to comprehend what you find then this is of little use. This is confirmed by research that shows that if you ask children to use words they have looked-up in a dictionary, you get statements like, “Mrs. Morrow stimulated the soup,” because one of the dictionary definitions of stimulate is stir-up.

But there is perhaps a more profound level to this argument. Cognitive load theory, which is based on a large number of empirical studies, posits a simplified model of the mind that consists of working memory – our conscious thoughts – and long-term memory. Working memory is severely constrained and we can only process about four items at a time. However, we can get around these constraints by drawing on schemas – webs of related knowledge – held in long-term memory. We can effectively process an entire schema without being subject to working memory’s constraints. So, knowledge in long-term memory is something you think with. It boosts your brain power. You cannot do that with knowledge held on the internet. Knowing stuff still matters. A lot.

Achievement boosts motivation

Many people seem to think that we need to motivate students to make them learn. This is often couched in terms of ‘engagement’. However, this idea may have cause-and-effect the wrong way around, or at least may neglect an important pathway. One Canadian study found that, for primary school children, maths achievement predicted later motivation but maths motivation did not predict later achievement. Pekrun and colleagues found more of a two-way relationship, but they still found a clear achievement-to-motivation pathway. This makes sense. Getting better at something is motivating.

Conversely, the classic approach of trying to excite students about, say, science by doing a funky demonstration, may not be as effective as we think. We may generate so-called ‘situational’ interest but this is unlikely to feed forward into a long-term interest in science on its own. Instead, we should probably investigate the most effective ways of teaching science – such as explicit teaching – to ensure students learn more, gain a sense of achievement and become motivated as a result.

Final thoughts

We often run up against a falsifiability issue in discussions of this kind. People have a tendency toward blanket rejections of whole swathes of evidence because it is based on test scores and they perceive these to be an invalid measure. However, if you are going to assert something to be superior, such as discovery learning, you should also be open to questions about what measures are capable of confirming your view or proving you wrong. If you cannot think of any, then you do not have an evidence-based position, you have a belief. That’s fine, but nobody else has to accept your belief or even pay it much attention.

And finally, some people on Twitter have suggested that tweeting the flashcard was just a transparent attempt at getting people to buy my book. This is not right. I actually have two books available and I encourage you to buy a copy of each. Or maybe a few copies. Or perhaps one for everyone in your organisation. That sounds like a good idea to me.

The Truth About Teaching is available to buy and The Power of Explicit Teaching and Direct Instruction is available to pre-order from all good online bookstores pic.twitter.com/LVZRvmmysu

— Greg Ashman (@greg_ashman) October 20, 2020