Cognitive load of rubbish?

I recently wrote a post on how knowledge of cognitive load theory (CLT) has changed my practice as a teacher. It was shared quite widely. I tend to attract critics of CLT and I think this is healthy. It is a relatively new theory and a reasonable and measured response is to take its finding provisionally. My own view of CLT is that the empirical studies are pretty robust but the theory is still under development.

When one fairly high profile person shared my post, Dr Sandra Leaton Gray responded on Twitter with some pointed criticism of CLT:

These criticisms immediately struck me as odd for two reasons. Firstly, CLT originated in Australia before a number of researchers from the Netherlands got hold of it. I wasn’t aware of many U.S. sources on the theory and had been under the impression that the U.S. was largely uninterested in it. And I seemed to recall John Sweller describing studies he conducted with Year 9 students, contradicting the suggestion that the sources are biased towards HE (higher education) and therefore the age group is too old.

I did a search for the original Sweller and Cooper studies on the worked example effect that sit at the root of CLT and found this paper. The subjects of this study were Australian (not U.S.) students from Year 9, Year 11 and university. Only Year 9 students were used for the critical tests of worked examples versus problem solving. The authors even discuss the fact that university students possess too much expertise in solving the required algebra problems for them to act as good test subjects.

The self-citation claim seems accurate but perhaps a little unfair. When there are only a small number of people working on a theory then who else should researchers’ cite? I think this was one of the reasons that Sweller and colleagues were pleased when CLT became popular among academics in The Netherlands.

Another issue that Leaton Gray raised was that the studies that form the basis of CLT are too old:

Again, this struck me as odd. Why does the age of the studies matter? Have children’s brain’s changed since the 1980s? That seems a short timescale for evolution.

And I am doing research right now into cognitive load theory with Australian school students. There are new CLT papers appearing in my various RSS feeds every day. If we look at the papers that form key points in the development of the theory then there is trail that leads right from the 1980s up to the present day. A quick search of Google Scholar with return relevant papers from every intermediate point e.g. this paper on the expertise reversal effect from 2003 or this paper from 2010 that demonstrates a worked example effect for an annotated Shakespeare play (participants: Year 10 students from Sydney).

Leaton Gray also suggested that neuroscience may somehow supersede the findings of CLT. This may be true but I am sceptical. I don’t think that neuroscience has much to say that is educationally useful and I’m doubtful if it ever will.

Finally, Leaton Gray’s argument migrated to the sample sizes used in CLT research:

I disagree with this for three reasons.

Oddly, really large sample sizes are friendly towards dodgy results. The way that calculations of statistical significance work means that if you make your sample size larger you are more likely to find a statistically significant result.

Secondly, small randomised controlled trials (RCTs) have their advantages: They are far less likely to be confounded. So far, the Education Endowment Foundation (EEF) have a strong record of testing the effect of doing nothing against the effect of doing something. Yet doing something usually involves a whole package of things. For example, if we give extra reading tuition then are we measuring the effect of the type of tuition or the effect of simply having more of it? Given the kinds of expectation effects that plague education research, we could find ourselves spending millions studiously measuring the effects of various kinds of placebos.

Finally, the EEF don’t usually randomise at the level of individual students. If you have 3000 students participating in a study across 30 schools and you randomise at the school level then the size of your sample is actually 30 rather than 3000.

Advertisements

9 Comments on “Cognitive load of rubbish?”

  1. Richard I says:

    “Educational refresh rate is 5 and a half years” …..

    Is she really claiming that anything she wrote before 2012 can be ignored.

    • Chester Draws says:

      Let us hope so.

      And, while we are at it, can we cease making any reference to Dewey (died 1952), Vygotsky (died 1934) and Freire (died 1997, but stopped publishing well before then).

      • Mike says:

        Oh, that sort of thing is priceless at the usual “expert” inservices, where the said expert is careful to demonstrate that the nonsense they are spouting is “evidence-based” nonsense. This is usually done with a footer on the PowerPoint citing “Noughboddie, X. (2013), Snaykoyle, Q. (2014), Tedtalker, Z. (2014)”, and then “Vygotsky, L. (1932)”. Always has me in stitches.

        The phrase “Educational research refresh rate is about 5 and a half years” is an absolutely iconic example of everything that’s wrong with contemporary university education departments.

  2. Stan says:

    No she is saying everything she writes and has written can be ignored because it may be superseded in the future.

    Clearly she is not going to win any critical thinking awards.

  3. Ann Williams says:

    For all aspects of WM + CLT, Susan Gathercole has written widely. Read: Gathercole, E. S., Woolgar, F., Kievit, R. A., Astlr, D., T., M., & J., H. (2016). How common are WM deficits in children with difficulties in reading and mathematics? Journal of Applied research in Memory and Cognition.

  4. Good rebuttal to some odd criticism, Greg. Twitter is the perfect idiom to take superficial pot shots at something. I would add to your points that cognitive load theory explains the success of explicit instruction nicely.

  5. Dylan Wiliam says:

    Anyone who criticizes a statistically significant result because the sample size is too small is just revealing that they do not understand very basic statistics. The real reason for being concerned about small samples is that they may not be random. And as George Gallup showed in the 1936 US presidential election, a small random sample is better than a large non-random sample.

    Moreover, as Greg points out, you also have to take into account clustering in the data. If the intervention is mediated by the teacher, then two students in the same class are not statistically independent of each other (they are more similar than two students in different classes). In fact for many educational experiments, 100 students, one from each of 100 schools would be a better sample than 1000 students from a single school.

    The other criticisms of CLT are equally odd. In fact anyone who actually reads Sweller, Kalyuga and Ayres’ 2011 book “Cognitive Load Theory” is likely to be impressed by the careful way in which the research evidence is built up. There are many grounds on which someone might critique CLT, but the fact that you do not like the results is not one of them.

    All this reminds me of a remark by Yale Law professor Ian Ayres. Ask any researcher for some research results that they believe, but don’t like. If they can’t come up with at least three, they’re probably a pretty bad researcher…

  6. eanelson2014 says:

    Sweller and Gathercole are highly esteemed scientists who study how the brain works. Sandra Gray is an education professor. On questions of how the brain works, should not the rule simply be: Science is what the scientific experts in a sub-discipline agree it is? And those who oppose and deny consensus science on how the brain works are arguing against scientific best practices, and therefore advocating mispractice if not malpractice? Should arguments that deny science be the basis for public policy?
    — rick nelson


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s