Differentiation: It is now time to think again

In January 2016, I wrote a blog post about differentiation. Part of the problem with differentiation has always been about what it actually means. In essence, differentiation is the attempt to cater to a wide range of students’ needs and abilities. Some people therefore class ability-grouping – placing students into different classes based upon their level of advancement – as differentiation, but in my experience, most people tend to restrict the meaning to catering to different students within the same classroom. All teachers do some of this, even just by setting extension work or answering students’ questions. However, the kind of differentiation advocated by education faculties and consultants tends to take this much further.

In the 2016 blog post, I referred to a graph I had generated in 2014 from PISA data showing that, if anything, greater levels of differentiation were associated with worse mathematics results and, referencing a piece of research from the U.S., I concluded that:

“Even when researchers have tried to make it work, they have complained that the teachers weren’t doing it right. So it is either something that works if you have particularly talented teachers who can implement it – although this has not been demonstrated – or it is an idea that doesn’t work at all. You decide. What is clear is that it is not an approach that is grounded in solid evidence.”

In May of the same year, I was surprised to find the same blog post referenced twice in an article in The Conversation by Linda Graham and Kathy Cologon titled, “Explainer: what is differentiation and why is it poorly understood?” The references to my blog post were not positive, they were used as examples to support the contention that differentiation is poorly understood.

In their article, Graham and Cologon start by noting that, “Differentiation is a long word that sounds complicated,” before suggesting that, “it just means teachers plan for the children who are actually in their class, instead of designing lessons for their idea of the ‘average’ child.” This implies that they do not think of ability-grouping as a form of differentiation and they explicitly criticise ability-grouping later in the piece. Oddly, they illustrate their understanding of differentiation with a clip from a Hollywood Movie before advancing a model of differentiation known as Universal Design for Learning or UDL:

“One particularly well-developed and internationally known resource is Universal Design for Learning (UDL). UDL is a framework informed by a large body of research investigating the many different aspects of teaching and learning involved.”

It seemed odd to focus on UDL because at least as late as 2014, the UDL website that Graham and Cologon linked to, in a reversal of the usual scientific method, was asking visitors if they could point to any evidence to support the various UDL provisions. And subsequent to the Graham and Cologon piece, a meta-analysis found that while UDL improves the ‘learning process’, the, “impact on educational outcomes has not been demonstrated.” Nevertheless, here were two education researchers confidently recommending UDL. On what basis?

In a sense, the evidence perhaps did not matter because Graham and Cologon issued a stern warning:

“Access and participation are the right of every student in Australia. The Disability Standards for Education are intended to support educators in understanding and implementing their obligations under national and international law. Differentiation is a requirement under these standards and necessary to provide quality education for all.”

I dispute that differentiation of the kind described in this piece is required by law. It is a conflation of the requirement to make ‘reasonable adjustments’ so that students with a disability may participate in class and the wider concept of differentiation. However, when I attempted to critique this piece, I was repeatedly referred to the Disability Standards (and sometimes even United Nations conventions) and Graham and her supporters interpreted my criticisms as a call to break the law. Graham and I blocked each other on Twitter around this time and Graham has since made a defamatory statement about this.

In the intervening years, my position has not changed, even as I have learnt of more evidence relevant to the debate. In 2018, I wrote a blog post for the Centre for Evaluation & Monitoring (CEM) in the UK where I made a number of points, including one on the ambiguity of what differentiation means:

“So ‘differentiation’ potentially covers a highly diverse range of practices; some we are pretty sure won’t work, some where the evidence is ambiguous and some that may be more promising. In this case, does the term have much utility?”

I had come to understand that completely opposite approaches, such as allowing a child to record audio rather than write an explanation versus giving the child intensive writing support, could both be described as differentiation. I expanded upon this argument for a 2018 Keynote at the Making Shift Happen conference in Amsterdam. And I revisit the argument for a chapter in my new book, The Power of Explicit Teaching and Direct Instruction.

I was also aware that Graham and colleagues were working on a meta-analysis that would, once and for all, demonstrate the effectiveness of differentiation. In 2018, in another piece for The Conversation, Linda Graham, Ilektra Spandagou and Kate de Bruin again advanced the cause of differentiation. This time it was in response to the “Gonski 2.0” review and the review’s recommendations that were supportive of differentiation. This time, the evidence cited was a study involving Middle School Science (which I have critiqued in my new book). When I questioned this evidence in the comments, asking why the authors relied on a study of this kind rather than a systematic review, Graham replied, “We are in the process of writing a systematic review ourselves.”

However, no such systematic review has materialised. According to the references in this study, a paper titled, “What exactly is Differentiation and why is it so poorly understood? A systematic review,” by Graham, Davis and Spandagou was presented in 2018 at a conference, but I cannot find the paper itself. A European Educational Research Association session also refers to a forthcoming paper, but interestingly also explains that there are many definitional issues in researching differentiation and a paucity of certain types of research.

Which brings us to a new paper that has been published, written by Graham, de Bruin, Lassig & Spandagou. It is labelled as a ‘scoping review’, although it uses a systematic method that covers papers published in the 20-year period from 1999-2019 from which the authors find 34 studies that meet their inclusion criteria. The abstract begins:

“The use of a pedagogical practice known as ‘differentiation’ has become more common over time as educators have sought to respond to increases in the diversity of students enrolling in their local school. However, there are now so many misperceptions and definitional inconsistencies that it is difficult to know what is being enacted in the name of differentiation or indeed what is being researched internationally.”

So, the definitional issues I have long identified are now being centered. And exactly how can we claim people have ‘misperceptions’ if the definition is all over the place? Who are the true holders the right perceptions? The authors?

The paper continues, telling a sorry tale of studies that don’t really demonstrate much about the effectiveness of differentiation. Most of the studies the authors could find are just surveys. Hardly any involve secondary school students. There are unsurprising findings such as that, “teachers who believe in the value of differentiation and its role in addressing equity concerns are more prepared to engage in it,” which do not address the question of whether differentiation works. Even then, the studies are, “undermined by methodological weaknesses.” The authors note that the, “…gap in the research on differentiation signals a potential problem, given the ubiquity of differentiation as a concept and its adoption by education systems and accreditation systems.” Well, yes.

The authors conclude that, “The diversity of focus… prevents comparison of findings and weakens the evidential basis to make claims of either differentiation’s effectiveness or indeed its ineffectiveness.” OK, so we haven’t found evidence of the Loch Ness Monster but neither have we found evidence that the Loch Ness Monster does not exist. So there’s that to hold on to.

This sequence of events, discussions and debates raises a number of points.

Firstly, we have gone from an ‘explainer’ about differentiation written by researchers to a point where the same researchers admit that there are actually a plethora of interpretations of what it means. Given their prior commitment to differentiation, Graham and colleagues should be commended for the intellectual honesty and integrity with which they have researched this question.

We also need to move on from the pantomime where anyone who questions differentiation is accused of incitement to break the law. It is now established that there may be legitimate questions and concerns.

And finally, why are we pinning so much on differentiation at the policy level when the evidence base is so thin? And why are we teaching trainee teachers that must differentiate? Why is it in our teaching standards?

I’ve been involved in this debate for at least six years. It is now time to think again.


Explaining Finland’s educational decline

Regular readers will be familiar with the phenomenon of Finland’s continuing decline on international assessments such as the Programme for International Student Assessment (PISA) run by the OECD:

Commentators seek to explain this decline in a number of ways. Pasi Sahlberg, former policy advisor in Finland and now with the Gonski Institute in Australia, attributes it to the distraction of digital devices (In August I debated Sahlberg on the educational lessons we may draw from COVID-19 and you can catch that debate here). And in this post, I critically examined the suggestion that a decline in effort from Finnish students is the cause.

A newly published PhD thesis by Aino Saarinen sheds more light on the possible causes. Saarinen examined data on Finnish students from the 2012 and 2015 rounds of PISA with a few questions in mind: Were scores correlated with self-directed learning practices, use of digital learning materials in schools or participation in an early learning programme? Also, was variance in learning outcomes, which is apparently increasing in Finland, associated with any of these factors?

Similar to findings made by the OECD themselves, as well as by researchers using OECD data to examine a number of different countries, Saarinen found:

“Frequent use of self-directed teaching practices or digital learning materials at school were associated with students’ weaker learning outcomes in several knowledge domains. Instead, frequent teacher-directed practices were related to students’ higher learning outcomes. Moreover, frequent use of self-directed teaching practices or digital learning materials had more negative impact on students’ learning outcomes in students with (vs. without) risky background.”

Saarinen found little impact from participation in early learning.

Of course, these results do not directly address the question of what has caused the decline. It is possible, for instance, that there was even more self-directed learning in, say, 2006. However, this seems unlikely given the qualitative accounts we have of how Finnish education has changed over the last 20 years or so (see e.g. here and here) and the more recent turn toward approaches such as ‘phenomenon-based learning‘.

And there are limitations on drawing too many inferences from the questionnaires that PISA uses to assess the practices that students are exposed to. They strike me as a little eccentric, with potentially overlapping concepts often present in different constructs. But then, nothing is ever perfect. It would be great to run a statewide randomised controlled trial, but that’s not going to happen and so we have to make our inferences as best we can from the data as it stands.

If we do consider the data as it stands, the discussion highlights – again – that constantly pointing to Finland as an example of educational excellence is flawed. It is a sign either of ignorance of the last fifteen years of data, self-deception or worse.


Emina McLean


Emina McLean is an Australian language and literacy expert. In this episode, she talks to Greg Ashman about her training as a speech language pathologist, her imminent career move to a brand new primary school in Melbourne, the science of reading and teacher professional development. Along the way, Emina and Greg discuss the controversies that surround literacy teaching, such as the recent blog post by Diane Ravitch complaining about the term the ‘science of reading’ and New South Wales’ decision to roll-out a phonics screening check in all public schools, as well as the best and worst ways of changing people’s minds and hopes for the future.


Absolute classic

Yesterday, there was an extraordinary rant on Twitter full of random CAPITALISATIONS by an American poet, translator, author and education professor called David Bowles. Here is a link to the thread. It is not viewable at present because Bowles seems to have protected his tweets, but you can see the river of effusive praise he received for taking such a bold stance. If you are wondering what it is all about, I had the following still cached on my phone (I have covered up the swears in consideration of those of you with a nervous disposition):


I think it is important to point out that the thread did not appear to be in response to anything in particular and the object of Bowles’ shoutiness appears to be all those of us who think that there is value in a canon of classic literature and that children should be expose to this canon at school.

I have a number of thoughts on this. Firstly, if we are proposing that there is some choice to be made between good contemporary literature aimed at young adults (YA) and children reading the classics, then this is a false choice. In my view, children should read the former for pleasure and the latter for class. Yes, we have a big problem with encouraging reading in young people. This is not solely the result of ignoring the evidence on systematic phonics, but paying attention to this evidence would be a good start.

I also think it is wrong to assume that children can only be interested in stories about people who look like them, are their age, live in similar neighbourhoods and have similar problems. Such books may be a gateway for some, but this idea ignores the wild popularity of escapist fantasy fiction. Children are capable of empathising with a wide variety of characters because children are human beings and empathy is our thing. If you ask a child whether they want to read Jane Austen, they will probably say, ‘Who’s Jane Austen?’. It is our job to open the world up to them and give them possibilities. Sure, they may dislike some or even most of what we serve them, but education is about expanding minds, not narrowing them.

And finally, there is the argument about core knowledge. Think of a book like 1984 and imagine being unfamiliar with its concepts. You wouldn’t understand what a person was talking about if they referred to a ‘Big Brother’ state. Some key texts function like this in our modern culture. A lack of such knowledge can be exclusionary. The kid who goes to university knowing nothing of Orwell will feel out of place in a first year lecture where such knowledge is assumed. I suppose the next logical step is to burn down the canon at the academy too.

And I do understand that for historical reasons, when you attempt to construct a canon of key literary works in a modern liberal democracy, your list tends to be overrepresented by white European men. So, yes, I think there is an argument for consciously trying to balance this, as I outline at the end of Chapter 1 of my new book, along with the mechanisms I propose for doing so.

However, there is perhaps a more fundamental point. Clearly, there are those who believe with capitalised, sweary fervour that it is far better to feed kids literary candy than proper books. I would not want them imposing this view on my children and I imagine they would not want me imposing my view on their children.

When positions on education are so diametrically opposed, when any possibility of rapprochement is eradicated by uncompromising rhetoric, is there any alternative to school choice?


Latest dispatches from the reading wars

Sarah Mitchell, the New South Wales education minister has announced the roll-out of a phonics screening check across NSW public schools in a robust article in the Sydney Morning Herald with the headline, “The reading wars are over – and phonics has won.” Great News. The phonics check is no panacea – and nobody is suggesting that it is – but we have found it very useful at my place and the findings of the pilot conducted in NSW this year are encouraging.

Mitchell makes the following point that made me wince as I imagined a few ‘balanced literacy’ advocates opening up their morning paper:

“Vice-chancellors need to take a broom to these faculties and clear out the academics who reject evidence-based best practice. A faculty of medicine would not allow anti-vaxxers to teach medical students. Faculties of education should not allow phonics sceptics to teach primary teaching students.”

A trip to Bunnings is on the agenda then.

Anyway, this put me in such a good mood that I decided to produce another one of my flashcards:

I have disagreed with Sarah Mitchell in the past on the topic of exclusions. However, it is not uncommon to find people who are pro-phonics and anti- school discipline – they usually work with children in a one-to-one setting. I guess some of these folks must be advising Mitchell.

It was only as few days ago that Diane Ravitch, once a profoundly sensible voice in the education debate, was objecting the the term, ‘science of reading,’ on her blog. It sounds, to Ravitch, much like talking about the, ‘science of cooking’. There I was thinking there was a science of cooking – an applied form of chemistry involving denatured proteins and the like. But, no, such a thing would be absurd! Good teachers are not scientists and good cooks are not scientists so there can therefore be no science of either. Got that?

However, while bending my head around this logic, I noticed something else. Ravitch mentions the late Jeanne Chall:

“Her 1967 book, Learning to Read: The Great Debate, should have ended the reading wars, but they continued for the next half century. She understood that both sides were right, and that teachers should have a tool-kit of strategies, including phonetic instruction, that they could deploy when appropriate.” [My emphasis]

Perhaps Chall’s 1967 book should have ended the reading wars and saved Sarah Mitchell the bother of doing it 53 years later, but it did not fit with my reading of Chall that she thought ‘both sides were right’. So, I grabbed my copy of Chall’s excellent 2000 book, The Academic Achievement Challenge. In this book, Chall writes:

“Several syntheses of the research comparing the effectiveness, for learning to read, of a meaning (whole language) versus a code emphasis (phonics)… found, in general, that classic approaches to beginning reading instruction (e.g. direct, systematic instruction in phonics – a code emphasis) were more effective than the various innovative approaches with which they were compared (e.g., a meaning emphasis, non phonics, incidental phonics, phonics only as needed, or a whole-language approach). The classic approaches were found to result in higher achievement in both word recognition and reading comprehension. They were more effective for different kinds of children and particularly for children at risk – those from low-income families, those of different cultural and ethnic backgrounds, bilingual children, and those with learning disabilities.”

That reads to me as if Chall had a firm view of which side was right and it is a timely reminder of why systematic phonics programmes are an equity issue.


A manifesto for an Australian College of Teachers

A recent tweet by Dame Alison Peacock, CEO of England’s Chartered College of Teaching reminded me of debacle surrounding that organisation:

I’ve watched it unfold from afar, from a failed crowd-funding campaign to fistfuls of UK government cash, an odd episode with a Russian TV channel and an election that resulted in the promised ‘teacher-led’ college being run largely by non-teachers. The only way the organisation could become worse is if it gained the power to regulate the profession. Teachers in England are almost universal in their criticism, with the main split being between those who still think its worth joining the organisation to change it from within and those who think it is beyond redemption. It clearly is beyond redemption and teachers must boycott this institution to deny it any credibility, making it harder for the UK government to give it any powers.

Nevertheless, it was not always obvious that the College would turn out this way. In the years prior to the crowd-funding campaign, there were some grounds for optimism. Teachers would value professional body focused on the practice of teaching rather than pay and conditions. And there is clearly a need to give voice to teachers, the most patronised and talked-over of the professions, which is why the fact that non-teachers muscled-in and took over the College caused such a deep psychic wound.

So, I began to think about what an Australian College of Teachers  – one that learnt the lessons of the English experience – would look like. Firstly, do we need one? Possibly, yes. There is a patchwork of state-based unions and teaching institutes in Australia but none of them really do this job. Our unions are far better than something like the absurd NEU in the UK with its ideological flights of fancy, but the focus of our unions is pay and conditions rather than teaching practice, as it should be. Our teaching institutes, on the other hand, are simply regulatory bodies. We hand over the cash and they give us permission to teach for another year. There is much to question in this regulatory model, but the fact that it would be pre-existing to any new Australian College of Teachers means that the probability of the new body being drawn into regulation would be low.

So, what would be the point? To test that out, I propose the following draft manifesto.

  • Membership of the Australian College of Teachers would be limited to those who currently teach classes in an Australian P-12 setting for at least three hours per week. This would include independent and government schools as well as specialist provision. Members who cease to meet this teaching commitment could continue as non-voting affiliates but nobody could join on that basis. School leaders, such as principals and deputy principals, and academics would not be barred from being members provided that they met the teaching load requirement. However, they would be barred from holding leadership positions within the College.
  • All leaders and officials would be drawn from the membership and explicitly not from any affiliates.
  • The proceedings would be conducted entirely virtually. There would be no physical meetings. The tyranny of distance in Australia is worse than in most countries and so if the College was based in a city and conducted in-person meetings, it would end-up being dominated by teachers from that city. By conducting all proceedings online, anyone with an internet connection could participate with no need to spend time travelling. A remote Northern Territory teacher could potentially lead the College. All meetings would be open to all members and affiliates as observers, making the College’s processes transparent.
  • The College would regularly survey its members (not affiliates) to establish the balance of membership opinion on matters such as workload, behaviour and the implications of government education policy. It would also commission a series of systematic narrative literature reviews on different aspects of teaching practice. This would improve on meta-meta-analysis by not trying to shoe-horn a complex issue into a single spurious measure. Members of the college would decide upon and pre-register the search and inclusion criteria and academics would produce the report which would then be peer reviewed. Different kinds of evidence such as correlational studies, quasi-experiments and randomised controlled trials could all be included in different sections of each report, with the authors suggesting overall conclusions which would be considered as advisory and provisional. There would be a rolling programme to update these reviews over time. These reviews would be published open-access on the College’s website.
  • When representing the College, leaders would be informed by the results of these opinion surveys and literature reviews e.g. “70% of our membership thought… A review we commissioned found…”
  • The leadership would seek to develop connections with journalists and organisations in order to 1) represent the position of the College in the media and in government consultations and 2) suggest teachers who would be available to give interviews and sit on conference panels in a personal capacity with the idea of injecting the views of actual teachers into education discussions.
  • The College would draw up a code of conduct based on ethical behaviour alone and membership could only be withdrawn on that basis. Any future attempt at turning it into a regulator of teacher competence or accreditation would therefore have to overturn these constitutional provisions.
  • Given the current climate, the constitution would also need an explicit commitment to free speech.
  • Service to the College would be voluntary and membership fees would therefore be low.

I am interested in your thoughts. Do we need such an institution? What do you think of my draft manifesto? What would you add? What would you change? Are there dangers I have not foreseen? Can any such body be designed in a way that avoids these dangers?

Also, to those of you in the UK who are familiar with the Chartered College of Teaching, what lessons can we learn?


Restorative Justice/Practices in Schools

Embed from Getty Images

I am not going to link to it, but there is a video currently going around Twitter that appears to show two schoolboys kicking and punching a Sikh boy in what seems to be a racially motivated attack. The video appears to have been recorded in England because the person who posted it to Twitter tagged the Twitter handle of an English school – which is one of the many reasons I am not sharing the tweet.

When it appeared on my timeline, I thought back to a segment that was broadcast last week on The Project, an Australian current affairs TV show.  I did not see the original broadcast, only the edit that was posted on Twitter.

In this edit, an Australian education academic makes the following claim about school suspensions and exclusions:

“There are other things that we could be doing that are supportive, things like Restorative Justice. So, for example, if one child hits another child, they’re going to learn more if they are required to sit and speak and apologise to the child that they’ve hit rather than, for example, if they’re sent home and they get to play X-Box for the day.”

I find this a strange perspective. Yes, it is important to identify learning opportunities for students, but we have to perhaps balance this against more pressing concerns. You won’t see anyone making a similar argument about, say, domestic violence and with good reason. Instead, our first thought is to protect the victim, with the rehabilitation of the offender a secondary concern. Yes, I do understand that there is a difference between adult offenders and children, but from a victim’s perspective, it’s as scary being a child attacked by another child as it is being an adult attacked by another adult. Sending the attacker home, whether he or she then plays X-Box or not, at least keeps the victim safe in the short term while emotions are still running high.

And what if the victim does not want to meet with their attacker? What if they find this prospect stressful? Do they have a say? Are there not other ways for the attacker to learn that hitting other children is wrong?

And some victims may be wary of taking part in one of these Restorative Justice meetings, sensing that the attacker may mouth an empty apology and then carry on as before when they leave the room.

I decided to ask Twitter for people’s experiences of Restorative Justice (sometimes known as Restorative Practices) in schools.

The thread continues to build, but one theme that has emerged is that the implementations of Restorative Justice that people tend to view positively still contain what some might view as a ‘punitive’ element. For instance, a student may be given a detention in which he or she is required to write about what they did wrong or a student who is temporarily excluded from class is required to have a discussion about the reasons why this happened when they return.

This suggests that framing Restorative Justice – or at least some people’s take on Restorative Justice – in opposition to suspensions and exclusions may be a false choice. Indeed, about 15 years ago when I was involved in suspensions in London, we would have a reintroduction meeting with the student and their parents where we developed a plan that included a report card, a set of targets to meet and a discussion of the supports the school needed to put in place to help the student meet those targets. So, you could describe that as restorative. However, I don’t think it would have occurred to me to involve victims in those meetings.

A number of comments in the Twitter thread suggest negative experiences of Restorative Justice and a few people have also contacted me privately to tell their stories. Here are a couple of examples that I have lightly edited to correct predictive text typos and the like:

“I have worked in a school with Restorative Justice and it was very difficult. In many cases it made no difference to persistent misbehaviour but undermined staff, who began to feel that they could not discipline students. Staff became highly stressed and disillusioned. [They were] Often told lessons weren’t well-planned or differentiated enough and that poor behaviour was their fault. At times staff felt they had no option but to apologise for reprimanding students who didn’t meet basic expectations. Students began to believe they were always in the right. I… left as a result.”

“Being a relatively new teacher I used to think restoratives were a progressive, democratic and effective method for helping kids change their behaviour. Instead, it’s turned into me losing upwards of an hour of my lunchtime throughout the week doing restoratives in which I just sit there. It’s often with the same kids, with no evidence of their behaviour changing at all. Many teachers at my school are hesitant to go through the process because it takes so much time and does not amount to much.

There have been a few cases where restoratives between victims and perpetrators have been held, and some of those victims have confided in me that they feel anxious during the meeting, and that they know the perpetrator is not sorry at all.

It’s a frustrating system that the kids can game, know that it isn’t a deterrent, and I, the other staff and kids suffer and learning is lost.”

Of course, these are just two accounts. It may be the case that the majority of implementations of Restorative Justice are better and it is the motivation caused by negative experiences that has caused these individuals to share their perspectives. That may also be true for the Twitter thread. And I have no means of independently verifying these stories. I doubt whether anyone is making stuff up, but we all emphasise the parts of a story we see as important and downplay the ones we see a less important or that perhaps run counter to the narrative that we want to put forward.

Interestingly, the item on The Project suffers from exactly the same problem. Children and parents with negative experiences were interviewed about their experiences of school suspensions with little obvious attempt made to independently verify these accounts.

Given the largely anecdotal nature of the discussion, it may be better to conduct wider, more rigorous research before adopting Restorative Justice as an alternative to suspensions and exclusions in schools. The evidence we currently have on Restorative Justice from the U.S. is not encouraging, suggesting that it may harm academic outcomes, harm the school climate as perceived by students and is difficult to implement properly, even with extensive support.


Greg Ashman with Kate Barry


In the episode, the tables are turned and Kate Barry, an English and French teacher from Ireland, interviews Greg Ashman about his new book, The Power of Explicit Teaching and Direct Instruction. Greg and Kate discuss Greg’s route into teaching, the nature and value of education research, the meaning of the terms ‘explicit teaching’ and ‘direct instruction’, the different perspectives of academics and practising teachers, the need to look for disconfirming evidence, differentiated instruction and solution to avoiding progressivist/traditionalist pendulum swings. Thanks to Kate for asking the questions. You can read an excerpt from The Power of Explicit Teaching and Direct Instruction here


How to frustrate maths students with mixed ability teaching

What would you conclude if, after persuading a school district to adopt your preferred model of maths education and studying a self-selected number of teachers in a couple of schools identified by the district as exemplars, you found that it wasn’t working out the way that you imagined and students told you that they found elements of the program frustrating? Well, you should perhaps conclude that there is a problem with your model. It would be a stretch, don’t you think, to blame the state’s maths standards? And yet that is the finding of a new study by LaMar, Leshin and Boaler which you can read open access in all its unfalsifiable glory.

Following the usual practice of one of the study’s authors, we are not told the real name of the district where the study was conducted. Instead, it is referred to as ‘Gateside’.

The U.S. has a different way of teaching mathematics to the Australian and English systems that I am familiar with. In the U.S., students typically follow a set sequence of maths courses: Algebra 1, Geometry, Algebra 2, Pre-Calculus and if they can fit it in, AP Calculus. This sequence means that students need to study Algebra 1 in the Eighth Grade if they are going to complete AP Calculus before the end of high school. AP Calculus is effectively, if not explicitly, an entry requirement for many top American colleges.

This tends to lead to a split where more advanced students take Algebra 1 in Eighth Grade, with their less advanced peers waiting until the following year. Gateside had decided to disrupt this model by making all students wait until Ninth Grade to study Algebra 1 and then teaching them in mixed ability classes. It would only be in Eleventh Grade that students could choose to either do Algebra 2 or Algebra 2 plus Pre-Calculus. The district also decided to eschew ‘procedural teaching’ in favour of a form of group work, the Platonic ideal of which the authors refer to as ‘complex instruction’. A dash of mindset theory was also supposed to be added somewhere.

The researchers approached the district and the district provided them with a list of seven out of their fourteen high schools that had, “fully implemented Complex Instruction and the district’s core curriculum.” The researchers then chose two of these schools to study. They approached the ten Algebra 1 teachers and eight of these agreed to participate in the study.

Apparently, one of the features of complex instruction is that students work together on ‘groupworthy’ tasks. The idea is that these tasks should only be possible to complete as a group. Moreover, every member of the group should be able to contribute in a substantial way to the overall solution and nobody in the group would finish the task before everyone in the group finished the task.

Clearly, such an idealised maths task is hard to design. Students who are more advanced at maths are going to be able to contribute more to a maths task than students who are less advanced. So, both groups became frustrated. The more advanced students were frustrated by having to constantly explain the maths to the less advanced students – i.e. by doing the teacher’s job – and by having to wait for them before they could move on. The less advanced students felt under pressure to work quickly. The teachers charged with managing the ensuing chaos developed a system where they would stamp the work so they could keep track of who had finished what and therefore decide when each group could move on. The researchers disapproved of these stamps.

This is a predictable clash between researcher idealism and teacher pragmatism. The researchers gave the teachers an impossible task. The teachers then tried to make this practical. The researchers then disapproved of the teachers’ solution.

The authors suggest that the main problem with the district’s approach – the one that caused all the frustration – lay in the tasks the teachers were setting and the fact that these tasks were driven by the curriculum standards. For instance, asking students to solve (2x+1)(3x+4)=0 is something the authors consider to be only a ‘beginning’ level of task. Finding the ‘zeros’ of the graph of y=(2x+1)(4x+4) is only slightly better and classed as ‘developing’. Instead, what we really want students to be doing is an ‘expanding’ task such as working out the pattern in a growing sequence of squares like this:

To be frank, this is the sort of task I would expect to see in maybe a Fifth Grade classroom. It has some connection to algebra, but not much. Most of the time that students are engaged in a task like this, they will not be developing their understanding of algebra. The first two tasks pay forward to later study of algebra, including calculus, in a way that the squares problem really does not.

So, if the state standards are forcing the teachers into tasks like the ‘beginning’ and ‘developing’ ones, they appear to me to be correct to do so. Attacking these standards seems like an excuse.

The real lesson from this study is something quite different: If you are able to persuade a district to adopt this model, they adopt it and then point you towards the best implementations of it, you will find it does not work. If that’s the case, what are the chances of getting this model to work at scale? I suggest they are very low.

It is worth noting that the researchers cite evidence that the new approach is superior than the old one. Apparently, the algebra failure rate dropped across the district. However, it is my understanding that U.S. schools do not use common, standardised assessments to determine who passes such courses and so this could simply be a function of applying a lower standard. We cannot check, because we don’t know the actual name of the district.

And the authors make a number of strong claims throughout the paper – claims that are potentially falsifiable such as, “When mathematics is taught as a set of procedures to follow, many students disengage, and various studies have shown that procedural teaching is particularly damaging for girls and students of color.” However, the reference provided is often then to another non-randomised study similar to the present one and involving some of the same authors.

One such claim that particularly drew my attention was that, “…procedural teaching encourages students to take a ‘memorization’ approach to mathematics, which has been shown to be associated with low achievement.” This refers to a paper from Scientific American that I have examined before and that I do not believe demonstrates this finding.

Nevertheless, people will point to this paper and districts will embark upon similar reforms, thinking they are evidence-based.