Cognitive load of rubbish?Posted: May 16, 2017
I recently wrote a post on how knowledge of cognitive load theory (CLT) has changed my practice as a teacher. It was shared quite widely. I tend to attract critics of CLT and I think this is healthy. It is a relatively new theory and a reasonable and measured response is to take its finding provisionally. My own view of CLT is that the empirical studies are pretty robust but the theory is still under development.
When one fairly high profile person shared my post, Dr Sandra Leaton Gray responded on Twitter with some pointed criticism of CLT:
These criticisms immediately struck me as odd for two reasons. Firstly, CLT originated in Australia before a number of researchers from the Netherlands got hold of it. I wasn’t aware of many U.S. sources on the theory and had been under the impression that the U.S. was largely uninterested in it. And I seemed to recall John Sweller describing studies he conducted with Year 9 students, contradicting the suggestion that the sources are biased towards HE (higher education) and therefore the age group is too old.
I did a search for the original Sweller and Cooper studies on the worked example effect that sit at the root of CLT and found this paper. The subjects of this study were Australian (not U.S.) students from Year 9, Year 11 and university. Only Year 9 students were used for the critical tests of worked examples versus problem solving. The authors even discuss the fact that university students possess too much expertise in solving the required algebra problems for them to act as good test subjects.
The self-citation claim seems accurate but perhaps a little unfair. When there are only a small number of people working on a theory then who else should researchers’ cite? I think this was one of the reasons that Sweller and colleagues were pleased when CLT became popular among academics in The Netherlands.
Another issue that Leaton Gray raised was that the studies that form the basis of CLT are too old:
Again, this struck me as odd. Why does the age of the studies matter? Have children’s brain’s changed since the 1980s? That seems a short timescale for evolution.
And I am doing research right now into cognitive load theory with Australian school students. There are new CLT papers appearing in my various RSS feeds every day. If we look at the papers that form key points in the development of the theory then there is trail that leads right from the 1980s up to the present day. A quick search of Google Scholar with return relevant papers from every intermediate point e.g. this paper on the expertise reversal effect from 2003 or this paper from 2010 that demonstrates a worked example effect for an annotated Shakespeare play (participants: Year 10 students from Sydney).
Leaton Gray also suggested that neuroscience may somehow supersede the findings of CLT. This may be true but I am sceptical. I don’t think that neuroscience has much to say that is educationally useful and I’m doubtful if it ever will.
Finally, Leaton Gray’s argument migrated to the sample sizes used in CLT research:
I disagree with this for three reasons.
Oddly, really large sample sizes are friendly towards dodgy results. The way that calculations of statistical significance work means that if you make your sample size larger you are more likely to find a statistically significant result.
Secondly, small randomised controlled trials (RCTs) have their advantages: They are far less likely to be confounded. So far, the Education Endowment Foundation (EEF) have a strong record of testing the effect of doing nothing against the effect of doing something. Yet doing something usually involves a whole package of things. For example, if we give extra reading tuition then are we measuring the effect of the type of tuition or the effect of simply having more of it? Given the kinds of expectation effects that plague education research, we could find ourselves spending millions studiously measuring the effects of various kinds of placebos.
Finally, the EEF don’t usually randomise at the level of individual students. If you have 3000 students participating in a study across 30 schools and you randomise at the school level then the size of your sample is actually 30 rather than 3000.