Why I’m happy to say that learning styles don’t exist

There is a curious constituency out there who are desperate to make the case for learning styles, despite all of the evidence that we now have. Some recognise the issues with the way learning styles have been implemented and yet still wish to keep them from the grave. We read that there might be something in it, somewhere; it’s all part of life’s rich tapestry.

This is a moral issue. Teacher workload is increased when managers pursue learning styles and we are all aware of the recruitment and retention problems affecting teaching. It is simply wrong to require teachers to fill-in boxes on lesson plans when it’s clear that there is no proven value in doing so. And yet this is an inevitable result of learning styles theories being undead. Just look at the responses to Tom Bennett’s recent tweets. The Zombie is out there and it is consuming the flesh of good, honest teachers. It is up to those of us with knowledge and a bit of a platform to try our best to kill it off.

Learning styles also risk labelling and stereotyping students; possibly the most dangerous part of the whole concept. As Coffield et. al. explain:

“The theorists warn of the dangers of labelling, whereby teachers come to view their students as being a certain type of learner, but despite this warning, many practitioners who use their instruments think in stereotypes and treat, for instance, vocational students as if they were all non-reflective activists… Similarly, students begin to label themselves; for example, at a conference attended by one of the reviewers, an able student reflected – perhaps somewhat ironically – on using the Dunn and Dunn Productivity Environmental Preference Survey (PEPS): ‘I learned that I was a low auditory, kinaesthetic learner. So there’s no point in me reading a book or listening to anyone for more than a few minutes’.”

The latest reanimation attempts seem to stem from two arguments. The Coffield et. al. study was one of the earlier reports to conclude that learning styles models offer little value (in the context of post-16 education). As part of the research, it found that certain learning styles surveys have more internal reliability than others (the same person would be assigned the same traits on repeated tests). That’s pretty much it. It certainly did not find that this is particularly useful and it recommended focusing on other strategies.

The other argument seems to be about the semantics of falsification. A scientific hypothesis is one which makes a testable prediction. Learning styles theories do this. If they are correct and there is some advantage to taking learning styles into account when planning lessons then we should see improved learning when we do this – this is the ‘meshing’ hypothesis. We don’t see such improvements. To most reasonable people, and to most scientists, this is seen as a falsification of the hypothesis (of course, it could perhaps be true that learning styles exist but have no implications for teaching whatsoever – I’ll leave you to ponder the logic and implications of that). Science uses inductive logic and therefore the scientific process of falsification must also use inductive logic.

The claim is that you cannot prove the non-existence of anything

The claim is that you cannot prove the non-existence of anything

Falsification does not meaning proving with absolute certainty that something does not exist because you can’t do this and it would therefore be impossible to falsify anything. If a psychic informs us that he has special powers then we may doubt this. We may ask him to demonstrate and he may fail. However, we cannot prove that he will always fail under all circumstances. Philosopher Steven Hale takes-up this theme in “Thinking Tools: You Can Prove a Negative“:

“Maybe people mean that no inductive argument will conclusively, indubitably prove a negative proposition beyond all shadow of a doubt. For example, suppose someone argues that we’ve scoured the world for Bigfoot, found no credible evidence of Bigfoot’s existence, and therefore there is no Bigfoot. A classic inductive argument. A Sasquatch defender can always rejoin that Bigfoot is reclusive, and might just be hiding in that next stand of trees. You can’t prove he’s not! (until the search of that tree stand comes up empty too). The problem here isn’t that inductive arguments won’t give us certainty about negative claims (like the nonexistence of Bigfoot), but that inductive arguments won’t give us certainty about anything at all, positive or negative.”

If we take such a strict line then it would be equally invalid to claim that fairies don’t exist or that Santa Claus does not exist or any strange and bizarre product of the human imagination. Perhaps we should be placing boxes on our lesson plans in which we explain how we intend to take account of UFOs or telepathy? Perhaps we should be incorporating tracts on reiki and homeopathy into trainee teachers’ reading lists? You never know, there might be something in it that teachers might find useful. There might just be a Bigfoot behind the next tree.

No, this is not a sufficient reason to ask teachers to spend valuable time on a concept. I admit that it is an inductive leap but I am happy enough to say that fairies don’t exist and I am happy enough to say that learning styles don’t exist either. And I am quite prepared to review this, if and when someone produces evidence that they do.

I will leave the final word on this to Pashler et. al.

“Our review of the learning-styles literature led us to define a particular type of evidence that we see as a minimum precondition for validating the use of a learning-style assessment in an instructional setting… we have been unable to find any evidence that clearly meets this standard. Moreover, several studies that used the appropriate type of research design found results that contradict the most widely held version of the learning-styles hypothesis, namely, what we have referred to as the meshing hypothesis… The contrast between the enormous popularity of the learning-styles approach within education and the lack of credible evidence for its utility is, in our opinion, striking and disturbing. If classification of students’ learning styles has practical utility, it remains to be demonstrated.”

Fairy Island

A small price to pay

I was on a bus passing through Brierley Hill on my way back home. It was the end of a long journey. A few weeks previously, I had been teaching physics in Kambuga. A couple of weeks after that, I had stayed in Masaka where I met a young man whose father had died fighting for Idi Amin and whose mother had died of aids, before travelling to Entebbe where I had heard the news that Princess Diana had been killed in a car crash. It was September, 1997.

There was something deeply strange about England at that time. It was as if I had flown in from Mars rather than Uganda. I had been troubled by the news of Diana but we hear troubling news all the time. You cannot avoid it. So I felt sad, but then I did what I had always done and carried on. Yet England was different. The shops in Brierley Hill High Street had signs saying “As a mark of respect for Diana, Princess of Wales, we will be closed on Saturday.” Everyone I met – people I had know for years – were full of an emotion that I didn’t feel connected to.

I began to understand what it must feel like to be a dissident; to not accept the popular narrative. I realised how close we are to unreason. And I realised how gossamer-thin our precious liberal democracy is. It wouldn’t take much for a demagogue to work out how to mobilise these emotions for their own ends. It has been done many time before; liberal democracies have evolved the wrong way into autocracies and, no doubt, they will do so again.

It is easy to think that you are protected from this if you grow-up in a country like Britain. We tend to think of freedom as an inevitability of history. Nobody will take away what has been so hard to achieve; what those before us have fought and died for. And yet there is no good reason to think this and, worse, the more we take it for granted, the greater the chance of it just slipping away, unnoticed.

We should always be on guard because people with power tend to act to limit and define the acceptable terms of any debate. South Africans should worry when somebody powerful like Blade Nzimande of the South African Communist Party says, “People can differ with me and you can insult me as you like, but disrespect, that is not acceptable.” How are we to define disrespect? Is it in the eye of the beholder? How are we to debate Nzimande if we disagree with his politics? Is it for him to dictate the terms?

Indeed, it is not hard to find examples from across the world of the powerful restricting the right to debate. It is not that they are against appropriate kinds of disagreement, they are at pains to explain, it is just that there are certain ways that people should go about it. In Kazakhstan, you must not “infringe the honour or dignity of the president” for instance and you might get arrested for the way that you protest against your local mayor. In Turkey you must not “publicly denigrate” the Turkish nation, military or police. Although, “expressions of thought intended to criticize shall not constitute a crime,” if that makes you feel any better. And many other nations have similar laws. Are they there to ensure civilised and respectful debate or to silence potential currents of opposition?

You may be feeling at this point that such legislation would not exist in a true liberal democracy like Australia or Britain, but it turns out that freedom is more of a continuum than a category. For instance, there’s a law in place in the UK that makes it illegal to use footage of Parliamentary proceedings in a comedic or satirical context. That’s odd, isn’t it? What possible point could such a law serve other than to prevent the powerful from looking foolish? 

I think that the right to satire is critical for a healthy political culture. It is a potent tool for exposing absurdity and hubris which is precisely why the powerful don’t like it.

This is why I am deeply suspicious of those with establishment roles and powers who try to restrict debate; those who try to use their authority to manipulate the ways in which people are allowed to challenge them; those who deliberately blur the lines between honest disagreement and abuse.

I have been thinking about the argument that we should not publicly disagree with fellow professionals or sneer at discredited ideas such as learning styles. I am struck that such contentions tend to come from those with traditional power and influence in our education systems.

Let me be clear. I am not drawing any kind of equivalence between unpleasant regimes and those who seek to restrict the way that we may debate education. I am trying to point out the tendency of the powerful to attempt to protect their position. It is quite possible to have an impoverished education debate in an otherwise open society. We have had it before and it would be a shame to return to it again.

Open disagreement is fundamental to reason. Some people won’t find it pleasant; like those British politicians who would prefer not to be sneered and jeered at. You can understand it. They probably find it disrespectful. But that’s a small price to pay for freedom.

Why educational theory is flawed

The TES has published an interesting piece by Janet Orchard that argues in favour of teachers learning educational theory. I think that educational theory is incredibly important and awareness of it among teachers is low. I also agree with this statement by Orchard:

“Teachers need to be able to plan successful lessons independently, and distinguish clear and legitimate aims from unclear and questionable ones. Teachers need to be able to communicate what they are doing clearly and coherently to parents and other stakeholders, justifying their professional judgements with legitimate and contextually relevant reasons.”

My concern is that a better knowledge of educational theory will not help teachers do this. It does not generally have this kind of practical value. Although very interesting, and perhaps essential to understanding the great debate in education, I tend to agree with Carl Hendrick’s assessment on Twitter that educational theory, “has little or no value for effective classroom teaching.”

Distilling the frenzy

The perceptive reader might wonder whether I have just contradicted myself. How can educational theory be both incredibly important and have little value for effective classroom teaching? It manages to do both by being largely wrong. And not just quietly wrong; educational theory is often loudly, ideologically, charismatically wrong. It sets the tone and manufactures the tropes through which educators express all kinds of misconceptions. It launches ships made of salt while the brass band plays and someone films it all for a TED talk. Others have invoked the words of Keynes when discussing educational theory. He is writing about economics but this quote is an excellent description of how educational theory affects our profession:

“The ideas of economists and political philosophers, both when they are right and when they are wrong, are more powerful than is commonly understood. Indeed the world is ruled by little else. Practical men, who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist. Madmen in authority, who hear voices in the air, are distilling their frenzy from some academic scribbler of a few years back. I am sure that the power of vested interests is vastly exaggerated compared with the gradual encroachment of ideas. Not, indeed, immediately, but after a certain interval; for in the field of economic and political philosophy there are not many who are influenced by new theories after they are twenty-five or thirty years of age, so that the ideas which civil servants and politicians and even agitators apply to current events are not likely to be the newest. But, soon or late, it is ideas, not vested interests, which are dangerous for good or evil.”

The scientific method

The problem with educational theory is partly due to what the term, ‘theory’, means when used in this way. In science, a ‘theory’ is the end-product of the scientific method. It has been rigorously tested against experiments or sets of observation (e.g. you can’t experiment on stars but you can conduct sophisticated observations). Scientists actively look for evidence that would prove their hypothesis wrong. We have a robust theory only after a hypothesis has undergone substantial testing and experimental results are replicated by other groups of researchers.

the scientific method

Educational theories, on the other hand, are often little more than what some eminent educationalist reckons. Sometimes they are supported by experimental evidence. However, it is often weak in nature and lacking in replication. Piaget’s stage theories have now been disproved even though he conducted plenty of experiments. The stages that he identified are likely to have been the product of the particular educational experience of the Swiss subjects that he studied (who included his own children).

Other educationalists are actively hostile to use of the scientific method. They talk of this as ‘positivism’ and apply special pleading. We are told that education is so terribly complex that scientific approaches break down. If it really is that complicated then it is hard to see how partial, unreplicated, uncontrolled, narrative accounts with a few numbers scattered around from time-to-time will have a better shot at uncovering the truth than scientific experiments.

Zombie notion

And yet the beast of educational theory lives.

Take Paolo Freire’s banking model. This is often trotted-out to explain why explicit instruction is such a bad thing. Yet explicit instruction is one of the educational practices for which there is strong empirical evidence. When you look at the case that Freire builds, it is clear that it is political; it is what Freire reckons about education based upon his revolutionary political beliefs and his experiences of teaching illiterate adults. It should be clearly seen as offering us little to apply in our K-12 classrooms.

Or there is the case of John Dewey. Almost a religious figure in the field of US education, his intentions and legacy are often fought-over. And yet his influence is at the heart of the ‘expanding horizons’ model of social studies education which has been turning students off the subject for nearly a hundred years. Despite being comprehensively critiqued in 1980, expanding horizons is the model for a brand new social studies curriculum in Australia.

All is not lost

But perhaps things are not as bleak as I suggest.

Psychological research is not perfect and it suffers from flawed studies and low rates of replication. Yet one of the reasons that I decided to pursue research in cognitive load theory is because it is one of the few frameworks that I have found to offer anything of practical use. Although often criticised as lab-based, much of the research has been done with students in classrooms. And it is unashamedly scientific in its approach.

Cognitive load theory is certainly not perfect and has stumbled along the way. So have other promising psychological theories. But if you want something of practical value in the classroom, modern applied psychology seems the better bet.

Aspirational Mathematics Education

I recently stumbled across the concept of ‘Ambitious Mathematics Instruction’. This term was applied by David Blazar to a set of indicators drawn from the Mathematical Quality of Instruction (MQI) classroom observation tool. It is a neat rhetorical device to give a positive name to a set of practices that you want to promote. Who wouldn’t want to be ambitious in their teaching of maths?

I started thinking about what I stand for. The term ‘explicit teaching’ is technically accurate but it’s a bit like launching a new brand of milk chocolate and calling it ‘cocoa and milk confection’. It’s not going to spread the love. Think about the rocks that are thrown at explicit instruction; its about compliance, drill-and-kill, transmission. It would be hard to be so casually dismissive of something with a fluffier motherhood-and-apple-pie name.  And while I’m defining it, I could throw in a few little fancies of my own that don’t strictly derive from explicit instruction. So let’s name it ‘Aspirational Mathematics Education’. This has the three-letter-acronym, ‘AME’, which sounds like ‘aim’ and therefore adds to the positive, aspirational connotations.

I think I’m on to something here.

Principles of AME

I have identified 14 principles of Aspirational Mathematics Education which I have listed below. I have added a link in brackets at the end of each statement where I am aware of research evidence to support it (some of this is a little oblique and some is repeated).

  1. A calm, positive and orderly classroom environment is required so that students may focus on mathematics (link)
  2. The classroom climate is such that students feel  mistakes are permitted and represent an opportunity to learn (link)
  3. Students practice recall of maths facts such as number bonds (to 20) and multiplication tables (to at least 100) in the early years of schooling to the point of instant retrieval. It is an expectation that all students can achieve this (link)
  4. Students are explicitly taught both concepts and standard procedures (link, link)
  5. Conceptual understanding is expected to influence procedural understanding and vice versa. The concepts underpinning procedures are fully explained to students, as is the procedural application of concepts (link)
  6. Students practice standard procedures to the point of fluency which is defined as follows: Students can perform the four basic operations on whole numbers (up to five digits), fractions and decimals with accuracy and automaticity (link)
  7. Teaching sequences start with teacher explanations and modelling followed by a planned, gradual release of teacher guidance (link)
  8. Teaching sequences start with highly similar examples and practice followed by a planned, gradual move towards more varied examples and practice that eventually cut across concepts (link)
  9. Key concepts and procedures are revisited many times in a year (link)
  10. Whole-class instruction is highly interactive. A whole-class segment sees the teacher call on students at random or request whole-class responses to questions. When demonstrating new procedures, teachers call on students to demonstrate steps within that procedure that are already known (link, link)
  11. Individual practice forms a key part of lessons, is valued and is seen as a means to correct misconceptions before they develop. Self, peer and teacher correction are used (link)
  12. Students frequently complete short, low-stakes quizzes (linklink)
  13. Correct, appropriately worked solutions are held to be the best evidence of learning although multiple strategies and student explanations would form part of classroom discussion
  14. Plans and resources are owned and shared across teaching teams and are refined in the light of assessment evidence. This is the main work of departmental teams

An aspirational model

So there you go. It is worth mentioning that I feel that I have some justification for labelling this as ‘aspirational’. Firstly, look at the built-in expectations of what students will know and understand. But also note the final point; an idea that could easily sneak past given that I can offer no supporting evidence. In Aspirational Mathematics Education, we wish to continually get better at what we do. We want to take last year’s plans and resources and use them as a starting point for this year. We want to build on the past, avoiding the mistakes and furthering the successes.

We want to create a ratchet rather than a wheel.


Is book sampling valid?

I recently read a post in which a senior member of staff described the process of book sampling in his school. Ideas like this spread like wildfire, especially when broadcast via the internet and by someone of such influence. Soon, everyone might be doing it.

It is worth noting that the blog comes from the UK where the pressures are different than other countries. The schools’ inspectorate there, OFSTED, sets the agenda for schools. Even if OFSTED do not ask for a certain practice, schools may adopt it, thinking that OFSTED would approve of it or under the mistaken impression that OFSTED have mandated it. This is perhaps exacerbated by the tendency for OFSTED inspectors to work with schools as consultants.

However, OFSTED can never be a good reason to adopt a particular approach. The only justifiable reason would be that it improves the quality of education; that it enhances learning. Book sampling as described seems to place onerous marking expectations on teachers and so there would need to be a clear benefit to it.

Research versus accountability

One key problem with book sampling is that a good research tool may not operate well as an accountability tool. For instance, imagine we conducted some research and found that the best teachers tended to spend more time standing at the backs of their classrooms. This observation might have some validity in helping to identify the best teachers. However, once teachers knew that managers were looking for this then even the weakest ones would start standing at the backs of their classrooms when being observed. This is a version of Goodhart’s law which is often paraphrased as:

“When a measure becomes a target it ceases to function well as a measure”.

We may protest that we are offering helpful, low-stakes feedback rather than holding teachers to account. Yet it is hard to interpret critical comments from a senior member of staff in any other way.

A poor proxy

It also seems to me that the quality of marking in an exercise book is a poor proxy for what students have actually learnt. This is even more pronounced in subjects with a large practical component such as PE or Drama because book work is only relevant to a part of the intended curriculum. There is research that the quality of feedback influences learning but this isn’t the same thing at all.

What if we conducted a sample and found a correlation between the teachers whose classes gained the most on external measures and our judgement of the quality of marking? Would this be convincing? Well, it would be circumstantial evidence but it would not imply cause. Perhaps the more effective teachers are better organised and therefore better able to implement the policy? The cause would then be teacher organisation and marking and learning would be two different effects of this. Improving the quality of a particular teacher’s marking would have little to no impact on student learning.

A possible relationship between quality of marking and student learning

A possible relationship between quality of marking and student learning

What is good marking?

Up to this point, we have assumed that we know what high quality book marking looks like. Usually, this is defined in terms of adherence to a school marking policy. Yet is it hard to trace a through-line from good educational research to the codes, multi-coloured pens, boxes and formalisms adopted by many schools.

If you consider what good quality book marking should achieve then you would perhaps arrive at a list like this:

  1. Provides useful feedback to the student
  2. Provides useful feedback to the teacher
  3. Is positively affective – makes the student feel that his or her work is valued

There are much more focused ways of providing feedback to both students and teachers than through the ritual of taking-up and writing in exercise books. You can use self and peer assessment in class before the lesson has finished. You can use a hinge point question in the middle of the lesson and reteach a concept if it is misunderstood. You can ask a question and get students to write their answers on a mini-whiteboard, immediately assessing what they can do and absolving the teacher of the task of writing the same thing in 30 books. If you want to assess an essay then get students to do this on paper and just take that up like you would for a test.

Although a small delay in feedback might sometimes be beneficial, I am aware of no studies that demonstrate the effectiveness of a smorgasbord of feedback that is given two weeks after the fact when the work has moved on to something else entirely. I doubt this sort of thing has much of an effect even if you ritually require the students to write a prose response. Yes, we should perhaps regularly revisit concepts but this would be better done by a recap or a quiz in class.

Homework is a particularly invalid thing to assess because we have no way of controlling the conditions under which it was done. Middle class students may get help and support from their parents that disadvantaged students don’t receive. Homework assessments would therefore reinforce a bias against the disadvantaged; a bias already identified in teacher assessment.

However, if all we wish is for our students to feel that their work is valued then we can periodically take-up exercise books to just read and sign. We can show we value homework by simply checking that it has been completed (perhaps at the start of the lesson) or by using the products of that homework in the following lesson.

A lazy teachers’ charter?

You might be feeling a little uncertain at this point. If we let teachers off-the-hook on the whole-school marking policy then won’t they just be lounging around drinking instant coffee and eating biscuits all day?

Book marking probably has a very low impact. Effective feedback should be specific and targeted rather than a diffuse mass of colours and codes imposed across subject boundaries. Freed from onerous marking policies, teachers could perhaps adopt more regular in-class assessment that focuses on the particular issues identified within that subject. They could spend more time analysing this data and looking for between-class differences that might identify more and less effective ways of teaching a concept. They could spend time feeding this quality information back into lesson and unit planning.

I reckon that would be better.

A Finnish tragedy

The night before researchED Sydney, I was at a dinner for some of the speakers. As part of this event, there was a badly-run panel discussion in which Geoff Masters of the Australian Council for Education Research (ACER) made the point that rather than looking at the current performance of countries in tests such as PISA and TIMSS, we should look at the direction of travel; improving or declining. This would strip out the effect of cultural factors.

Unfortunately, this seemed to pass right over the heads of members of the audience who proceeded to enthuse over Finland. Finland is a favourite of progressive educators because they project on to it all sorts of things that they approve of whilst ignoring contrasting evidence from other countries that also do well in PISA and TIMSS. Perhaps these folks were simply unaware that Finland has been declining in its PISA performance over roughly the last ten years.

To perhaps address this decline, the powers that run Finnish education have decided to introduce a cutting edge approach known as ‘phenomenon-based learning’. It is pretty hard to distinguish this from William Heard Kilpatrick’s ‘Project Method’ which he first published in 1918.

A new episode of The Educators on BBC Radio 4 looks into this. Sarah Montague interviews Helsinki’s Education Manager, Marjo Kyllonen. We discover that cross-disciplinary phenomenon-based learning is all about ownership of the learning, that teachers have to give up control and that learning must be relevant and connected to the students’ lives. “We don’t want to change the child but we want to change the school,” she tells us. A student talks of qualified enthusiasm but comments that, “I wouldn’t want it to last a whole year… there’s still that traditional education that gets the message through.”

However, Kyllonen has wider ambitions. Even within disciplines she would like teachers to prioritise relevance and student exploration. To me, this sounds like inquiry learning. We hear that today’s students are somehow different from students in the past and that the new approach will enable them to learn marketable skills such as collaboration, social skills, critical thinking, creativity and cross-disciplinary thinking.

If you are familiar with my blog then you will know that I am sceptical that such attributes are ‘skills’ that can me improved through training. For instance, Dan Willingham has written about the fact that critical thinking can’t really be taught. And anyone can be creative; what society values is people who can be creative in interesting and useful ways. Newton was creative but his creativity stood at the apex of a growing body of knowledge of which he was an expert. “If I have seen further, it is by standing on the shoulders of giants,” he wrote. Quite.

Lose the traditional disciplines that best represent what we know in a coherent way and you risk the discoveries and innovation of the future.

Creative Crap (own work)

Creative Crap (own work)

However, the BBC have done their research. In addition to talking to a concerned Finnish academic, they interview Tim Oates who has written a superb paper on Finnish educational performance. If we want to understand the performance of 15-year-olds in 2000 then we need to look at the education that led up to that and to a process of education reform that began in the 1960s. This was characterised by inspections, grade-level testing and state-sanctioned textbooks.

Sarah Montague challenges Kyllonen for the evidence to support phenomenon-based teaching. It doesn’t appear that there is much and Kyllonen is certainly not keen to use tests to measure the effects. If there were any evidence then it would have already shown up at some point since 1918. This is nothing new and it has been tried many times before.

It is odd that a country with a good track record, that has the answers to the problems that it wishes to solve in its own history, is so keen to strike out along a century-old, ideologically-driven dead-end. It’s a tragedy.

Evidence that inquiry maths works

A piece of econometric research by David Blazar was recently brought to my attention on Twitter (thanks to @dylanwiliam via @drlindagraham). It seems to show that, “inquiry-oriented instruction positively predicts student achievement,” in maths. In other words, the more inquiry that teachers use, the better the students’ maths performance.

This conflicts with the evidence that I tend to present which shows that explicit instruction is more effective than inquiry learning and so I was intrigued.

I cannot comment on the complicated statistical methods used (I’m still learning how to do a t-test). However, I noticed a few things worth commenting on.

The evidence that inquiry learning works

The study analyses a number (N=111, I think) of Grades 4 and 5 teachers working in three unnamed districts in the eastern US. Blazar has clearly gone to a great deal of effort to deal with the fact that this is not a randomly controlled trial. Students are not randomly assigned to teachers and there are school effects and so on. The fancy statistics are there to take account of all this.

When done, Blazar finds a positive effect size of about d=0.1 for something called ‘ambitious mathematics instruction’ which he links with NCTM reform maths and ‘inquiry-oriented instruction’. He also finds a negative effect for teachers’ mathematical errors and imprecisions and, surprisingly, very little effect for things like classroom climate and behaviour management.

Ambitious mathematics instruction? What’s that?

Interestingly, Blazar only uses the term ‘inquiry’ in the abstract, introduction and discussion. The proxy for inquiry that he measures in the actual study is a construct known as ‘ambitious mathematics instruction’; an odd, approbatory name for a set of teacher characteristics. This derives from a classroom observation instrument known as the Mathematical Quality of Instruction (MQI).

I looked-up the MQI and found that it didn’t mention the term ‘ambitious quality of instruction’ at all. Instead, the items grouped under this heading in Blazar’s paper are grouped under several different heading in the MQI instrument itself. Rereading the paper, I realised that Blazar had contributed to an earlier study that showed that the items on the MQI cluster into two main factors; ‘ambitious quality of instruction’ and ‘mathematical errors and imprecisions’. So Blazar is using a construct that he was involved in developing. Unfortunately, I don’t seem to have access to the study that shows this clustering.

Blazar’s claim is that these factors are closely related to both the NCTM standards and the new Common Core standards. This is interesting because I’m not sure that NCTM and Common Core are meant to be quite the same things. In addition, when you look at what is grouped under this heading, it is not clear that these are all elements of inquiry-oriented instruction. Explicit instruction would have equal claim to ‘Linking and connections’, ‘Explanations’, ‘Generalisations’, ‘Math language’ and ‘Remediation of student difficulty’.

A focus on ‘Multiple methods’ and ‘Student explanations’ would certainly be more of a preoccupation of reform maths than explicit instruction, particularly if formally assessed. However, these would still be part of the repertoire of explicit teaching, although maybe less so than in a reform classroom. I wasn’t sure what ‘Use of student productions’, ‘Student mathematical questioning and reasoning’ and ‘Enacted task cognitive activation’ meant so I went to the MQI itself. The first seems to be about the teacher interacting with the students and the last two seem to be the closest fit to standard meanings of ‘inquiry’ because they involve students posing mathematical questions, identifying patterns and so on.

How was the teaching measured?

I have already noted that teaching was scored according to the MQI rubric. This was done by videoing each teacher three times and then scoring the videos. The teachers knew when they were being videoed because they were allowed to choose the dates in advance. Blazar notes that they would have had no incentive to vary their instruction for these observations and quotes evidence from the MET project that indicated that rankings were similar whether teachers knew in advance about an observation or not. However, I am sceptical about this argument. A position in a ranking is different to the balance of learning strategies used in an individual lesson. And the ranking would stay the same if everyone’s performance dropped by the same amount for surprise observations.

Reform maths has been promoted in the US since the release of the NCTM standards in 1989 and so I think the ‘right’ way of teaching maths would be pretty clear to teachers with typically nine years of service. Despite having no incentive, it is natural for teachers to want to look good and so I can imagine them emphasising reformy elements in these taped lessons. Presumably, the more able teachers would be differentially more aware of and able to do this.

Did teachers know the MQI criteria on which they would be judged?

Indeed, a lot of psychology research depends upon the principle that people will expend effort without an incentive. Consider a typical randomised, anonymous trial with a post-test. There is no incentive at all for students to try-hard on the post-test. They could simply give nonsense answers. And yet we routinely use evidence from such studies when comparing instructional approaches.

How was performance measured?

Maths performance was measured using a test that Blazar notes was ‘developed by researchers who created the MQI’. This is interesting. We might expect a greater correlation between higher teacher performance on the MQI and higher student performance on a test developed by the same people because they would both prioritise the same things. For instance this test is a “cognitively demanding test,” in which items previously analysed by researchers, “often asked students to solve non-routine problems, including looking for patterns and explaining their reasoning.”

It would therefore seem likely that teachers that emphasise these aspects of maths will prepare their students better for these tests.


Given my previous writing on the matter, I am obviously biased against a result that shows a positive impact of inquiry-learning. So perhaps I am not the best person to draw a conclusion here. Instead, I will point out the obvious contrast to another econometric study with a different methodology, call for replication of this research and suggest that you make up your own minds. But please, read the paper first. You can follow the arguments even if you don’t follow the statistics.

inquiry maths