The evidence for Dan Meyer’s TED talk

Those of you who are familiar with this blog will be aware that I have highlighted differences between my position and that of Dan Meyer. In particular, I have criticised some of the ideas that Meyer presented in his 2010 TED talk. The claims that he made can be typified by this extract from the transcript:

“So 90 percent of what I do with my five hours of prep time per week is to take fairly compelling elements of problems… from my textbook and rebuild them in a way that supports math reasoning and patient problem solving. And here’s how it works. I like this question. It’s about a water tank. The question is: How long will it take you to fill it up? First things first, we eliminate all the substeps. Students have to develop those, they have to formulate those. And then notice that all the information written on there is stuff you’ll need. None of it’s a distractor, so we lose that. Students need to decide, “All right, well, does the height matter? Does the side of it matter? Does the color of the valve matter? What matters here?” Such an underrepresented question in math curriculum. So now we have a water tank. How long will it take you to fill it up? And that’s it.”

Meyer goes on to talk of his use of video to create engagement with the problem. It is clear that he sees motivation as important. He also speaks as if these are ideas that he has developed and evaluated himself, in his own classroom – “It’s been obvious in my practice, to me.” – rather than anything that is the product of educational research. He does not refer to evidence that his is a more effective form of teaching. This would be fine if he was simply presenting ideas for us to contemplate but, instead, he is making a strong claim; that we need a general change to the way that we teach mathematics.

I have queried this lack of evidence before and I have suggested that it is a good example of the problem with how we talk about education. It is hard to imagine this kind of a discussion about the practice of any other profession (if we can class teaching as a profession – this might disqualify it).

In my view, Meyer’s ideas are at odds with cognitive load theory. The type of instruction he describes might be effective for students who are already relatively expert but the evidence suggests that it is unlikely to work with novices who haven’t had much instruction in the topic. A lot depends upon the enactment. If the open-ended task comes at the end of an extended period of instruction or it is brief and is followed by comprehensive, explicit instruction then this might work reasonably well. If we are hitting relative novices with such problems then it would seem to provide too little guidance.

In a recent exchange with Meyer on Twitter, I suggested that the difference between us was that I had evidence for my position and that he did not have evidence for his. He did not agree with this and when I asked for the evidence to support the claims in the TED talk, he offered the following:

I thought that I should look these up. They are not full references and so I entered the terms into Google Scholar and read the first paper that seemed relevant. I expected the evidence to be about problem-based learning because that is the way that I would broadly categorise Meyer’s method.

I was familiar with Mayer’s work on multimedia and it seems that this supports Meyer’s suggestions about using multiple forms of presentation, although it must be pointed-out that textbooks with diagrams do also fit the definition of ‘multimedia’. It seems sensible to use arresting graphics, visuals and video as long as we pay heed to cognitive load theory and its implications which, according to Mayer, include:

“(1) the presented material should have a coherent structure and (2) the message should provide guidance to the learner for how to build the structure. If the material lacks a coherent structure — such as being a collection of isolated facts — the learner’s model-building efforts will be fruitless. If the message lacks guidance for how to structure the presented material, the learner’s model-building efforts may be overwhelmed. Multimedia design can be conceptualized as an attempt to assist learners in their model-building efforts.”

This would not seem to support the idea of using distracting information with novice learners and I would argue that many textbook questions have substeps in order to assist learners in their model-building efforts. [I will also take this opportunity to add that Mayer has written an excellent paper on the repeated failure of discovery learning]

The paper that I found by David and Roger Johnson on ‘constructive controversy’ doesn’t seem to have much to do with the effectiveness of maths teaching, focusing largely on variations of social studies lessons. It does have some data and reports effect sizes. However, many of the dimensions on which constructive controversy is assessed are things like attitudes and motivation. When academic performance is reported, constructive controversy seems to lead to ‘higher-level reasoning strategies’. I think it would be important to look at exactly how this is measured, particularly since much of the discussion is about understanding the position of opponents. It is not obvious how this might apply in maths.

It also seems that  constructive controversy is not compared with explicit instruction in these studies (the comparison conditions are group ‘concurrence-seeking’, ‘debate’ where a judge decides on winners and losers and ‘individualistic efforts’). I am willing to concede that controversy may make something memorable and interesting but I am not sure that we have to follow the Johnson’s model in order to introduce it or that explicit forms of instruction cannot make use of controversy.

And so, finally, I reviewed an article by Kasmer and Kim on ‘The nature of student predictions and learning opportunities in middle school algebra.’ I can see that prediction might be a component of problem-based learning and that it might help to activate prior knowledge or give knowledge-gap experience to students. Yet the study in question seems to have no control group and so we can’t really establish the extent of the advantages of predictions over any other type of instruction (I did briefly look for controlled studies but couldn’t find any):

“In order to investigate the value of using prediction, prediction questions were developed and posed in one middle school mathematics classroom throughout one school year. The target content area was algebra: linear and exponential relationships.”

Let us assume that there are significant advantages to making predictions. This still does not imply the use of problems with distracting information and substeps removed. Predictions could as easily be incorporated into a period of explicit instruction as one of problem-based learning. We could ask students what they think will happen before demonstrating the correct method. In fact, I do this a lot.

In my view, the papers that I have presented do not provide compelling evidence that maths teachers should follow the approach to maths teaching that Meyer describes. Instead, there’s seems to be a much stronger case for using explicit instruction. You can find some of the evidence to support my view here and I intend to have an ebook available soon where I will discuss this at greater length.