In January 2016, I wrote a blog post about differentiation. Part of the problem with differentiation has always been about what it actually means. In essence, differentiation is the attempt to cater to a wide range of students’ needs and abilities. Some people therefore class ability-grouping – placing students into different classes based upon their level of advancement – as differentiation, but in my experience, most people tend to restrict the meaning to catering to different students within the same classroom. All teachers do some of this, even just by setting extension work or answering students’ questions. However, the kind of differentiation advocated by education faculties and consultants tends to take this much further.
In the 2016 blog post, I referred to a graph I had generated in 2014 from PISA data showing that, if anything, greater levels of differentiation were associated with worse mathematics results and, referencing a piece of research from the U.S., I concluded that:
“Even when researchers have tried to make it work, they have complained that the teachers weren’t doing it right. So it is either something that works if you have particularly talented teachers who can implement it – although this has not been demonstrated – or it is an idea that doesn’t work at all. You decide. What is clear is that it is not an approach that is grounded in solid evidence.”
In May of the same year, I was surprised to find the same blog post referenced twice in an article in The Conversation by Linda Graham and Kathy Cologon titled, “Explainer: what is differentiation and why is it poorly understood?” The references to my blog post were not positive, they were used as examples to support the contention that differentiation is poorly understood.
In their article, Graham and Cologon start by noting that, “Differentiation is a long word that sounds complicated,” before suggesting that, “it just means teachers plan for the children who are actually in their class, instead of designing lessons for their idea of the ‘average’ child.” This implies that they do not think of ability-grouping as a form of differentiation and they explicitly criticise ability-grouping later in the piece. Oddly, they illustrate their understanding of differentiation with a clip from a Hollywood Movie before advancing a model of differentiation known as Universal Design for Learning or UDL:
“One particularly well-developed and internationally known resource is Universal Design for Learning (UDL). UDL is a framework informed by a large body of research investigating the many different aspects of teaching and learning involved.”
It seemed odd to focus on UDL because at least as late as 2014, the UDL website that Graham and Cologon linked to, in a reversal of the usual scientific method, was asking visitors if they could point to any evidence to support the various UDL provisions. And subsequent to the Graham and Cologon piece, a meta-analysis found that while UDL improves the ‘learning process’, the, “impact on educational outcomes has not been demonstrated.” Nevertheless, here were two education researchers confidently recommending UDL. On what basis?
In a sense, the evidence perhaps did not matter because Graham and Cologon issued a stern warning:
“Access and participation are the right of every student in Australia. The Disability Standards for Education are intended to support educators in understanding and implementing their obligations under national and international law. Differentiation is a requirement under these standards and necessary to provide quality education for all.”
I dispute that differentiation of the kind described in this piece is required by law. It is a conflation of the requirement to make ‘reasonable adjustments’ so that students with a disability may participate in class and the wider concept of differentiation. However, when I attempted to critique this piece, I was repeatedly referred to the Disability Standards (and sometimes even United Nations conventions) and Graham and her supporters interpreted my criticisms as a call to break the law. Graham and I blocked each other on Twitter around this time and Graham has since made a defamatory statement about this.
In the intervening years, my position has not changed, even as I have learnt of more evidence relevant to the debate. In 2018, I wrote a blog post for the Centre for Evaluation & Monitoring (CEM) in the UK where I made a number of points, including one on the ambiguity of what differentiation means:
“So ‘differentiation’ potentially covers a highly diverse range of practices; some we are pretty sure won’t work, some where the evidence is ambiguous and some that may be more promising. In this case, does the term have much utility?”
I had come to understand that completely opposite approaches, such as allowing a child to record audio rather than write an explanation versus giving the child intensive writing support, could both be described as differentiation. I expanded upon this argument for a 2018 Keynote at the Making Shift Happen conference in Amsterdam. And I revisit the argument for a chapter in my new book, The Power of Explicit Teaching and Direct Instruction.
I was also aware that Graham and colleagues were working on a meta-analysis that would, once and for all, demonstrate the effectiveness of differentiation. In 2018, in another piece for The Conversation, Linda Graham, Ilektra Spandagou and Kate de Bruin again advanced the cause of differentiation. This time it was in response to the “Gonski 2.0” review and the review’s recommendations that were supportive of differentiation. This time, the evidence cited was a study involving Middle School Science (which I have critiqued in my new book). When I questioned this evidence in the comments, asking why the authors relied on a study of this kind rather than a systematic review, Graham replied, “We are in the process of writing a systematic review ourselves.”
However, no such systematic review has materialised. According to the references in this study, a paper titled, “What exactly is Differentiation and why is it so poorly understood? A systematic review,” by Graham, Davis and Spandagou was presented in 2018 at a conference, but I cannot find the paper itself. A European Educational Research Association session also refers to a forthcoming paper, but interestingly also explains that there are many definitional issues in researching differentiation and a paucity of certain types of research.
Which brings us to a new paper that has been published, written by Graham, de Bruin, Lassig & Spandagou. It is labelled as a ‘scoping review’, although it uses a systematic method that covers papers published in the 20-year period from 1999-2019 from which the authors find 34 studies that meet their inclusion criteria. The abstract begins:
“The use of a pedagogical practice known as ‘differentiation’ has become more common over time as educators have sought to respond to increases in the diversity of students enrolling in their local school. However, there are now so many misperceptions and definitional inconsistencies that it is difficult to know what is being enacted in the name of differentiation or indeed what is being researched internationally.”
So, the definitional issues I have long identified are now being centered. And exactly how can we claim people have ‘misperceptions’ if the definition is all over the place? Who are the true holders the right perceptions? The authors?
The paper continues, telling a sorry tale of studies that don’t really demonstrate much about the effectiveness of differentiation. Most of the studies the authors could find are just surveys. Hardly any involve secondary school students. There are unsurprising findings such as that, “teachers who believe in the value of differentiation and its role in addressing equity concerns are more prepared to engage in it,” which do not address the question of whether differentiation works. Even then, the studies are, “undermined by methodological weaknesses.” The authors note that the, “…gap in the research on differentiation signals a potential problem, given the ubiquity of differentiation as a concept and its adoption by education systems and accreditation systems.” Well, yes.
The authors conclude that, “The diversity of focus… prevents comparison of findings and weakens the evidential basis to make claims of either differentiation’s effectiveness or indeed its ineffectiveness.” OK, so we haven’t found evidence of the Loch Ness Monster but neither have we found evidence that the Loch Ness Monster does not exist. So there’s that to hold on to.
This sequence of events, discussions and debates raises a number of points.
Firstly, we have gone from an ‘explainer’ about differentiation written by researchers to a point where the same researchers admit that there are actually a plethora of interpretations of what it means. Given their prior commitment to differentiation, Graham and colleagues should be commended for the intellectual honesty and integrity with which they have researched this question.
We also need to move on from the pantomime where anyone who questions differentiation is accused of incitement to break the law. It is now established that there may be legitimate questions and concerns.
And finally, why are we pinning so much on differentiation at the policy level when the evidence base is so thin? And why are we teaching trainee teachers that must differentiate? Why is it in our teaching standards?
I’ve been involved in this debate for at least six years. It is now time to think again.
