A new test of ‘productive failure’Posted: November 17, 2016 Embed from Getty Images
Productive failure is a concept that has become popular in recent years, particularly among maths teachers. It derives from a number of studies by Manu Kapur in Singapore that have been disseminated through articles and blog-posts aimed at teachers and the general public. Productive failure follows in the footsteps of similar strategies such as invention learning or introducing desirable difficulties.
Advocates of productive failure accept the need for explicit teaching. However, they propose that students should be allowed to struggle a little and attempt to solve problems for themselves before they are explicitly taught a solution method. There are three mechanisms that they propose for why productive failure is beneficial:
- By first grappling with the problem, students might better recognise its deep features.
- Attempting to solve the problem activates the prior knowledge networks within students’ minds (often known as ‘schemas’) so that new learning fits into these networks appropriately.
- Students become aware of the gaps in their knowledge and may become more focused on these gaps or more motivated to close them during later teaching.
There has been some criticism of the experiments that have been used to support productive failure. These studies have not always varied one factor at a time (see questions at the end of this article). When they have been more rigorously designed, these studies don’t always replicate real-world forms of instruction. For instance, in a 2014 study, it seems that students in the control condition were taught a solution method and then asked to solve the same problem in as many ways as possible. It is doubtful that an ordinary maths teacher would use such an approach. And there has been conflicting evidence – studies where the advantages of productive failure have not materialised or appear more nuanced.
A different criticism is theoretical. Cognitive Load Theory has been successful in explaining many learning related phenomena and generally predicts that, for novice learners, fully guided instruction and the use of worked examples is superior.
This tension therefore makes for an interesting area of research because there are two theories that make different predictions.
And so it was with interest that I read a new paper that tests the predictions of productive failure. The paper is by Likourezos and Kalyuga and it is worth pointing out that Slava Kalyuga is one of my PhD supervisors. The study was carried-out with high school maths students who were learning a geometry topic over a six week period. I have summarised the design in the diagram below:
There were eight regular maths lesson that were split into two 30 minute phases. At the outset, students were randomly allocated into one of three conditions where, for the first 30 minute phase of each lesson, they either received fully guided instruction in the form of worked examples, partially guided instruction that included some scaffolds or unguided problem solving. The second half of each lesson consisted of explicit instruction to the whole group. There were 24 students in each condition and the pre-test showed no significant variation between the groups.
The researchers basically found no difference between conditions in a later post-test. This test consisted of two parts: a component containing questions that were very similar to those used during the teaching, as well as a component that consisted of slightly different kinds of questions – ‘transfer’ questions. There was no effect on either component.
When they analysed the data further, they did find a few statistically significant results. Perhaps surprisingly, students in the fully guided condition produced a greater number of creative (and still correct) solutions on the post-test; solutions that involved pre-empting a more sophisticated method that is often taught to older students. Students in the fully guided condition also showed a significantly greater level of interest and felt they were more likely to be successful than those in the unguided group. Students in the unguided condition reported significantly higher levels of challenge than those in the fully guided group.
If these results have not occurred by chance then it seems to me that there are two viable explanations. The first is that the advantages of productive failure and the advantages of providing worked examples trade-off against each other, resulting in no overall effect. Another possibility is that the effect of the explicit teaching component is so large that it washes out the effect of the previous conditions. In this case, the worked examples condition might be redundant because it is effectively repeated in the explicit teaching phase.
Either way, this result is important because it represents a key attempt to replicate previous productive failure findings. The lack of replication should prompt us to pause before making further recommendations for the use of productive failure.