Is ‘metacognition and self-regulation’ an actual thing?

When I google the term ‘metacognition and self-regulation’ (in quotes), the top hits are from the UK’s Education Endowment Foundation (EEF) and Evidence for Learning, an organisation that uses and promotes EEF materials in Australia.

I’ve long argued that this term is a problem because it seems to contain a bunch of things that are largely unrelated to each other. My interest was piqued when I spoke to someone from the EEF who was quite critical of Nick Gibb, England’s school minister, and Gibb’s focus on phonics and subject knowledge. Instead, EEF evidence was apparently pointing in the direction of ‘metacognition and self-regulation’.

The EEF use ‘effect size’ as their main measure of an educational intervention; something they then translate into additional months of progress that the intervention will supposedly provide. This is flawed because it’s not appropriate to compare effect sizes for studies with different types of outcome test (experimenter designed versus standardised), different subject matter, different age groups, different cohort types (selective or full ability range) and different study designs.

You see this effect in the EEF results; its own randomised controlled trials (RCTs) often result in lower effect sizes than studies using weaker methods. Nevertheless, it’s worth looking at the data that goes into the EEF’s measure of metacognition and self-regulation. I’ve summarised the studies, plus one additional study, in the table below:

The studies in italics are ones that the EEF have drawn from a review of the available literature. They typically consist of meta-analyses of a range of studies that are not all RCTs. I have stated the kind of outcome that was assessed, as best I understand it, in brackets.

Below these are the studies carried out by the EEF. The final EEF study listed is for, ‘Let’s Think Secondary Science’, an intervention based upon the longstanding Cognitive Acceleration in Science Education (CASE) materials. The EEF have not yet included this result in their overall weighted average for metacognition and self-regulation, but they should. ‘Let’s Think,’ is actually very similar to the, ‘Thinking Doing Learning Science’ intervention for primary students which is the subject of the other Hanley et al. study in the table.

For completeness, it’s worth sharing that the two Gorard et al. references reflect the two different outcomes of the (in)famous Philosophy for Children trial and the NIESR references reflect a study involving mindset. Torgerson et al. evaluated an explicit writing intervention.

Given this information, how should we interpret the EEF’s claim that, overall, implementation of a metacognition and self-regulation strategy will result in an additional 8 month’s progress and is, “High impact for very low cost, based on extensive evidence’?

Firstly, apart from the explicit writing intervention evaluated by Torgerson et al., school leaders should probably avoid the rest of the EEF interventions. Even the one other clearly statistically significant result, with an effect size 0.22, maps to far less than 8 additional months progress (I make it about 3 months according to the EEF conversion chart).

We may achieve a larger effect if we implement one of the non-EEF interventions. However, it was the problems with the methods used by these kinds of studies that prompted the foundation of the EEF in the first place.

The second, and perhaps most fundamental, point that the table highlights is the disparate nature of the trials and outcomes. What does an explicit writing intervention have in common with a mindset intervention or philosophy for children?

Explicit attempts to teach reading and writing strategies have a long track record and are clearly defined, even if the former may provide a limited, one-off benefit. Would it not aid clarity to give these interventions their own categories? After all, we should not assume that adopting a mindset intervention will lead to the same effect as an explicit writing intervention.

Not only are these programmes qualitatively different to the others in the metacognition and self-regulation basket, the mechanism seems far more obvious. It is entirely plausible that teaching children to plan their writing will improve the structure of their writing and lead to higher writing scores. But how is Philosophy for Children supposed to improve maths?

The EEF seem strangely attached to evaluating interventions with obscure or mysterious mechanisms. For instance, in evaluating a core knowledge programme, they taught children about one domain of knowledge and then gave them a reading test in a different domain. Not even enthusiasts for core knowledge would argue this should work because the whole idea is that reading is enhanced by knowledge of the domain, not by knowledge of a different domain.

I wonder whether researchers at the EEF subscribe to a form of genericism, where academic performance is viewed as the result of a set of general, trainable skills. That might explain the way they design some of their studies, as well as their attachment to the metacognition and self-regulation chimera.

For their part, school leaders need to be aware that this category is deeply misleading, that it isn’t an actual thing and that they need to look under the hood if they want to find anything worth examining.

Advertisements

11 Comments on “Is ‘metacognition and self-regulation’ an actual thing?”

  1. I’ve often argued that ‘metacognition’ (being cognizant of one’s own cognition) is an irrelevant term as we also have the term ‘cognition’ and that is sufficient. I’ve also argued that it is repetitive or even recursive because if one is aware (cognizant) of one’s own metacognition, this becomes meta-metacognition and… ad infinitum.

    • Michael Pye says:

      Love that idea! We could use indices to differentiate the depth of our thought. “Thinking about this problem on a meta^4 cognitive level i have deduced”. Algebraically we could define a hypothetical level of thought of level meta^x cognition. Defining x as infinite would clearly mean it is capable for the human mind to reach godhood. I could produce an unintelligible book explaining this process. Its genius being that it could be interpreted at any level from meta^0 to Meta^whatever. But wait what about Cantor’s theories around infinity. Maybe there is a even higher level of thought?

  2. Tom Burkard says:

    Good post. You don’t often see Alice in Wonderland in modern primary schools, probably because it’s too close to the bone.

    Our friend Carol Torgerson has form. In 2006 she led the DfES fightback against synthetic phonics, claiming that

    “There is no strong RCT evidence that any one form of systematic phonics is more effective than any other… Since there is evidence that systematic phonics benefits children’s reading accuracy, it should be a part of every literacy teacher’s repertoire and a routine part of literacy teaching, in a judicious balance with other elements”.

    To call this disingenous would be charitable. In the US–the source of the evidence–the term “synthetic phonics” does not refer to the radical measures outlined in Rose; Torgerson was merely comparing programmes which had varying degrees of emphasis on teaching synthetic and analytic phonics within an eclectic framework.

    In any case the RCT model, however suitable it may be for evaluating new drugs, is less than ideal for assessing new teaching methodologies which are inconsistent with teachers’ training. In an analysis of a Michigan reading initiative, Standerford found that such changes as teachers actually implemented were seldom inconsistent with their prior preferences or beliefs about the best way to teach reading. Another problem is the biases of the researchers, which may affect the design and methodology of the study.

  3. Stan says:

    Greg,
    I think you are happy that we can be more or less aware of how our cognition is working and that regulating things like cognitive load is helpful.
    So its seems to be something very specific you are objecting to here. That is that teaching self awareness and self regulation is not as effective as someone else monitoring and regulating what is going on.

    A bit like saying if you want to improve an athlete’s training worry more about what the coach knows about training than the athlete knows about it.

  4. Chester Draws says:

    Haven’t teachers always taught children to reflect on their learning? I was at school 40 years ago, and a very traditional school at that, and I was taught to think about what I was doing and why I was doing it.

    I expect my students to spot their particular areas of weakness, to learn to pace themselves, to fight through mental barriers. To do this they need help, so I am explicit to them about what they need to work on. (It’s literally hopeless to try to expect a child to find their blind spots themselves, because they are blind to them.)

    What traditional teachers don’t do is fetish this.

    Or is “meta-cognition” somehow deeper than the very old-fashioned “reflecting”?

  5. Didier says:

    I tend to struggle with these terms ‘metacognition’, ‘leaning to learn’ and ‘self regulation’, but i do relate much more to the term reflexivity which is from what i understand is more about applying external tools, methods and concepts to our own cognitive actions. I don’t expect my students to make good use or progress through ‘metacognition’ unless I inform them about cognitive facts like the absence of multitasking ability, the benefits of retrieval practice, interleaving or spacing effect but then i advice them to adapt their practices, not their mental processes.

  6. […] The IoE debate focused on the possibility of evidence-informed practice. Let me be clear, all of the contributors had something valuable to say on this topic. Even though I disagree with Gert Biesta’s position on the question, his views are valid and worth considering. And I was amused by the contribution of Sir Kevan Collins, chief executive of the Education Endowment Foundation (EEF), with his swipe at people with large numbers of Twitter followers – who could he possible mean? – and his attempt to insist that the EEF has no agenda (when it clearly has). […]

  7. […] If you want to get a flavour of the article that I will write, if accepted, then take a look at this recent post. […]

  8. […] such as the Education Endowment Foundation in the U.K. and it’s agenda to promote ‘metacognition and self-regulation‘, and wonder what agendas might capture this new […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s