Where is the evidence to support differentiation?

Following my last post, a number of people challenged me on Twitter about differentiation. I was not surprised by this. Differentiation is an article of faith within schools of education and questioning it is seen as heresy.

Let me make clear that I am not against all forms of differentiation. Part of the problem is the elasticity of the term. I have had discussions with some who regard the practice of placing students in different classes based upon prior achievement as ‘differentiation’. The evidence for ability grouping is ambiguous but at least it has the potential to be efficient. I also employ practices that could be described as ‘differentiation’ in my own work. For instance, I often ask some students to work on problems while I reexplain a concept to a section of the class.

Yet to many teachers and teacher educators, differentiation means something quite specific. It means grouping students within a class and planning different activities for these groups as well as perhaps providing options for students. For instance, this article in The Conversation describes a process of differentiation that allows students to demonstrate what they have learned through alternative means:

“Actually, differentiation is about teachers providing choice to avoid discriminating against students who may be disadvantaged by ‘one size fits all’ approaches…

It does mean that students with expressive language difficulties or dyspraxia, for example, who experience difficulties with writing are given the opportunity to demonstrate their learning in other ways.

This was famously demonstrated in the movie The Blind Side where Michael Oher’s teachers conducted oral assessments, enabling him to successfully demonstrate his learning.”

I think this is a flawed idea. If students struggle with writing then doing less writing than their peers hardly seems like a plan to address this. Over time, if this kind of differentiation is the norm across classes then we should see a widening gap in writing ability between the child with language difficulties and his or her classmates. Perhaps we intend to just give writing away? I’m not prepared to do that.

I am also of the view that within-class differentiation is impractical to plan and inefficient to deliver. It requires a teacher to manage multiple groups engaged in different tasks. In a sixty minute lesson with a class divided into five groups, the teacher is able to spend a maximum of twelve minutes with each group. And that’s before you take out all the time spent just managing the arrangement; “The table at the back need to focus while I talk to these guys at the front. Thank you. Wait, why are you wandering around, Carl?” These factors are likely to be part of the problem with the L3 programme in New South Wales.

So I have reasoned arguments against differentiation and I think they are valid ones. However, I also have some empirical evidence. A large scale study was carried out across a number of states in the U.S. One of the authors of the study was Carol Ann Tomlinson of the University of Virginia, probably the world’s greatest proponent of differentiation. So you would imagine that they gave differentiation a pretty good go. Yet they couldn’t get it to work. This may be because it was hard to train teachers to do it correctly, it may be because differentiation is flawed in principle or it may, as I suspect, be a bit of both. It’s not promising, however you look at it.

If we accept that whole class teaching is the default and that differentiation involves teachers doing something different, then the burden of proof actually lies with differentiation’s proponents. This burden appears even greater when you consider that every school of education teaches differentiation as best practice and it is required by the Australian Professional Standards for Teachers. You would think there must be tons of evidence. So it should be straightforward – my arguments should be easily falsified by citing studies that show that differentiation leads to enhanced educational outcomes.

The most popular framework for differentiation in Australia seems to be Universal Design for Learning (UDL). I have my doubts about this model but an advocate of differentiation pointed me to a paper with the promising title of, “The effectiveness of universal design for learning: a meta-analysis of literature between 2013 and 2016.” Perhaps this would conclusively prove me wrong? Perhaps it would demonstrate clear evidence for the effectiveness of this form of differentiation? The abstract states:

“Universal Design for Learning (UDL) is often promoted as an inclusive teaching methodology for supporting all students within diverse contemporary classrooms. This is achieved by proactively planning to the edges of a classroom by thinking of all the potential needs of students. To examine its effectiveness, a meta-analysis was conducted on empirical research, containing pre- and post-testing, published in peer-reviewed journals between 2013 and 2016 (N = 18). Results from this analysis suggest that UDL is an effective teaching methodology for improving the learning process for all students. The impact on educational outcomes has not been demonstrated. The implications of this study will be discussed.” [my emphasis]

The paper lists many positive experiences of using UDL and positive effects on how teachers and students feel, but the evidence for impact on educational outcomes is indeed lacking. This is a problem because it is possible to make students and teachers feel positive about pretty much any innovation. What we really need to know is that it leads to more learning.

It was also suggested to me that lots of teachers use differentiation every day and find it effective. How can I dismiss these teachers and their experiences? Do I not value their opinions? Unfortunately, testimonials are an unreliable form of evidence. If I were able to produce a list of teachers who said differentiation does not work for them – which I reckon I probably could – then would this refute such an argument? I suspect I would be told that they must not be doing it properly.

Finally, I was linked to a piece that attempts to explain why teachers are sometimes resistant to differentiation. It’s fascinating. I was particularly taken by this claim:

“Research clearly indicates that students will typically perform better on standardized tests when they have had the opportunity to learn in preferred modes, even if the test is not in their preferred mode. “

Unfortunately, there is no link to this research or a reference to follow. Preferred modes of learning are known more commonly as ‘learning styles’ and this statement expresses the ‘meshing hypothesis’ that matching instruction to a student’s preferred learning style is more effective than not matching the instruction in this way. This hypothesis has been extensively investigated and no supporting evidence has been found.

There are plenty of educational strategies that, like differentiation, lack a solid evidence base. That’s not necessarily a problem. Teachers should be presented with these strategies and a fair appraisal of the research. They may then – individually or as part of a whole-school approach – make informed decisions about which ones to apply.

What I object to is the enforcement of differentiation through teacher standards and the misleading claims that it is best practice or that teachers who don’t apply a particular form of differentiation are discriminating against groups of students. That is coercive. It might be justified if the evidence for the effectiveness of differentiation was strong but it clearly is not.

Advertisements

31 Comments on “Where is the evidence to support differentiation?”

  1. […] via Where is the evidence to support differentiation? — Filling the pail […]

    • Neil MacKay says:

      Differentiation works for me – especially differentiation by outcome which is regularly slated by academics – often called “lazy teaching” by many who should know better – but when properly done empowers bright but vulnerable students, especially Dyslexics damaged by the system, to show what they know in alternative ways . Then it becomes the springboard to writing in formal assessments. This is not to “give writing away” but rather to acknowledge that if we teach the way we always do, some students will give us what they always give us. Empowering students to show they know via mind maps, flow charts etc and then “talking” the plan is a brilliant launch pad for writing. One size definitely does not fit all – especially when it assumes that only kids who can read, write and spell accurately are smart. My experience suggests that many students with good basics do not have an original thought in their head. Have a look at Tom West’s recent posts about the failure of valedictorians in the USA system to make any mark whatsoever in later life

  2. If I explain the same principal in two or three different ways = differentiation
    If I provide resources that explain the same thing in different ways = differentiation
    If I provide resources that have written, visual and video explanations = differentiation
    If I use different metaphors that range from simplistic to sophisticated = differentiation
    If I provide choices in assignment topic = differentiation
    If a student can get a S or a pass in the topic with a variety of means = differentiation

    I don’t believe that differentiation means splitting the class into groups etc – if that is accepted as the default definition of differentiation the argument has already been almost already lost.

    I would proudly say “I differentiate in my classes”, give examples like those above and invite anyone unhappy to prove that they are not examples of differentiation.

    • Michael Pye says:

      This is the problem with definitions they evolve or in this case generalize. The differentiation movement focused on the second and third of your examples as these were perceived to be different from prevailing practice. Teachings habit of co-opting names but trying to change their meaning to what we already do frustrates clarity and muddies our ability to develop. Changing the meaning of differentiation now is an attempt to rob the word of its power to cause immediate damage (not a silly approach admittedly) but sacrifices the war to win the battle. There will be another poorly evidenced idea coming soon and we need to improve our professions ability to spot and reject it.

    • Mitch says:

      I hope I’m not putting words in his mouth but I think Greg would take the most issue with your 5 and 6.
      Obviously there is always variability between student responses and different strengths and weaknesses, but when students have the opportunity to avoid those weaknesses, they do so how do they improve those weaknesses? As Greg and others have pointed out, it can often lead to an increase of the gap rather than a decrease.
      Like most teachers and schools, I use the medically diagnosed line. I’m not completely happy with some of the diagnosed conditions but that is well above my paygrade and understanding. If it’s not – I’m probably not catering for it.

      • For point #6, take VCE, specifically Yr 11 (Units 1&2). The VCAA produces lots of assessment advice along the lines of

        “The award of satisfactory completion for a unit is based on a decision that the student has demonstrated the set of outcomes specified for the unit. Teachers should use a variety of learning activities and assessment tasks that provide a range of opportunities for students to demonstrate the key knowledge and key skills in the outcomes.”

        Note that achieving an “S” (~ a “pass”) is different to the level of achievement, or a grade. It simply demonstrates that the key knowledge/skills have been attained.

        Then for point #5 – say for a particular aspect of key knowledge you were going to assess via a research assignment. Clearly within the broad overview of the key knowledge there would be assignment topics which would be more straightforward and others more sophisticated –
        any of which could earn the student an “S”.

      • Mitch says:

        There is a difference between using a variety of learning activities and assessment tasks, and allowing a large variety of responses for an assessment task.
        Presumably the variety was chosen because there is worth in assessing that subject’s knowledge and skills in those contexts.

    • Tempe says:

      In my child'”s primary school differentiation does indeed mean splitting the class up into ability groups and giving them different material to work on. There are four groups each doing different things according to their level. I do think this can be very bad for the morale of the student (ie I’m in the dumb group) and that it’s really difficult to prove that you can move up if you are only given a certain level of work. The bottom end kids wont have access to the higher end stuff so when it comes to testing time they will probably only be capable of answering the bottom end questions. I prefer high school where they have an extension class for those high ability kids and the rest are in a mixed class with a range of abilities. These kids all access the same content and are tested on the same content.

      I do think that there should be measures to ensure the bottom end kids are grasping the material. I think this is a part of the teachers job. ie wander around the class room to assist when needed or ask the class questions or set homework based on revision of the lesson. I also think that in primary school if a child is really struggling in reading or maths then they should have access to help via being taken out of regular class and given remedial help.

    • Erik Devlies says:

      Could you please explain “If a student can get a S or a pass in the topic with a variety of means = differentiation”? It’s the only one I don’t understand. Coudl be my English or difference of education system…
      Thank you.

  3. Stan says:

    Differentiation should only be done when both students and teachers understand their limits. Probably continuity too.

    • Michael Pye says:

      Confused! I never now my limits accurately, my students less so.

      • Stan says:

        It’s a math joke.

        But if you do think of calculus and the shape of any benefit curve of attention to one item as a percentage of your maximum attention you would expect the benefit to rise and then flatten and then decline and eventually become negative. The most benefit comes well before this is the only thing you attend to.. At some point you are doing enough that something else would be more benefit and the opportunity cost outweighs the benefit.

      • Stan says:

        This is the real reason understanding differentiation is important. If you don’t know whether the benefit of doing more of something is positive or negative you can easily go wrong.

      • Michael Pye says:

        Groan! I get it now with your explanation.

  4. JL says:

    They used to talk about special education students needing to be in the “least restrictive environment”. What was never discussed is that ALL students should be in a “least restrictive environment”. Insisting on “mixed ability” or “inclusive” classrooms you are guaranteeing that that is unachievable. Learning STYLES may have been debunked (and I’m glad!) but there are still differences in the pace at which students are able to learn and the amount of repetition that individual students need. Ability grouping by classes is simply the most effective (not perfect) way to be able to do whole class teaching. It’s not discrimination. There should always be the ability to move up or down. And ideally it should be by subject. A student could very well be advanced in math but need extra help/time with writing. Some students will take longer to get to the same destination–that they get there should be far more important than that they did so in a specific amount of time or that they stayed with their age group. Inclusive classroom ideology has given me classes with students reading at a 4th grade level in with student reading at a college level but earning the “same credit” for a second language class. Differences in background knowledge and processing speeds DO make a huge difference. inclusive classes and “social” promotion have resulted in the “achievement gap” and more of the same won’t close that gap.

  5. Gvginkel says:

    Differentiation is at least one of the most difficult teaching skills to develop. It does appeal to predict student engagement (at least, the specific behaviours listen in this article) http://www.sciencedirect.com/science/article/pii/S0191491X14000595?via%3Dihub but engagement is only a proxy for learning.

    • Chester Draws says:

      but engagement is only a proxy for learning.

      And not a very good one.

      I would argue that parental income is probably about as good a proxy for learning as engagement, in that I could just are reliably pick someone’s marks based solely on their parent’s income as I could based on how engaged they are.

  6. Hi Greg, just a couple of queries or clarifications related to the above that 140 characters allow.

    Would you consider that your stance and framing is more related to a HS context than a Primary Context?

    Secondly, one of your criticisms of differentiation is around the practice or proposition that students are given the opportunities to demonstrate their knowledge in ways other than writing. If I am correct, your contention is that this only serves to reduce the amount of writing done by students who need more writing perhaps, than less? If a student has difficulties with writing that limit their ability to demonstrate knowledge (particularly in a learning area such as Science, Humanities or Health), does that mean that we value that knowledge less because they can’t express it in writing?

    In regard to the above, I consider it would be highly appropriate to implement a number of potential alternatives to allow them to demonstrate the knowledge or skills in this other learning area in a non-written format. If we asked them to write an essay or information report or whatever text as a demonstration of the subject knowledge and skills, what we would see is a representation of their English, as opposed to their science/humanities etc. These students would still be required to write and develop their skills in English components and, in many cases, there is more focused effort to develop and improve their skills and target their ‘weaknesses’ or developmental needs.

    Another example would be in mathematics where the heavily word-oriented nature of some mathematics assessments (including NAPLAN and other standardised instruments) impacts on the ability for an accurate representation of a student’s mathematical knowledge and skills. I have seen (I know that my observation isn’t ‘evidence’) that there is a direct correlation between reading comprehension skills and performance in some of these tests. Whilst we target their reading comprehension skills, knowing how they can support their engagement across the curriculum, it would be reasonable (IMHO) that I seek/allow alternative ways to demonstrate/represent their mathematics.

    I know you didn’t give this specific aspect as much treatment in your blog, or in twitter commentary as perhaps some other aspects, so thought I would extend the query here.

    Thanks in advance for you clarification. Whilst I mightn’t agree with everything you write, I am interested in understanding your thinking.

    • Chester Draws says:

      If a student has difficulties with writing that limit their ability to demonstrate knowledge (particularly in a learning area such as Science, Humanities or Health), does that mean that we value that knowledge less because they can’t express it in writing?

      You are conflating two things and assuming they have equal value. They don’t. The first is their knowledge. The second is their ability to express that knowledge. In general the public value the second only. So the answer to your question is, in general, yes.

      A musician who plays beautifully in private but cannot perform in public is not going to be hired by an orchestra. A person who can understand scientific principles but not solve problems using them is no use to anyone much. A person with great insight into society but cannot write a book/report/article to express that insight might as well not have had the insight, given that it is trapped in them.

      My experience, and it is only my experience, is that this is mostly a non-problem in practice. Students with natural ability can be taught to write, speak, perform if they are sufficiently motivated to do so. That some people with natural ability in a subject can’t write essays is usually a function of their lack of interest in doing the hard yards of learning how to write essays, not some underlying issue that cannot be overcome. I don’t see that the “solution” to their lack of motivation is to sidestep the problem and pretend it doesn’t exist.

      • Hi Chester, we are talking about 6 and 7yo in my case. I think that you are thinking about it from a far too narrow perspective.

        Yes, the ability to communicate knowledge in written form is a specific skill. But there is underpinning knowledge that is represented in this.

        Take a 7yo in my class currently. His skills in written communication are low. But he has a strong grasp of the scientific principles of evaporation. If I ask him to write an explanation of this process this knowledge will not be reflected. If I ask him to orally explain it to me, model it or draw a diagram, he can do so very effectively.

        In your music example, perhaps that musician isn’t going to be hired by an orchestra but could be an outstanding session musician who works in a recording studio or composes work. If he/she doesn’t perform in public, does that mean they are not a talented musician?

        Finally, I cannot at all get my head around this quote from your reply:
        “That some people with natural ability in a subject can’t write essays is usually a function of their lack of interest in doing the hard yards of learning how to write essays, not some underlying issue that cannot be overcome.”

        Thanks for taking the time to comment.

      • Chester Draws says:

        OK Steve. I come from a High School perspective, which is quite different.

        With little ones, who cares if they can’t display knowledge that they have? Unless you are in the French system, where they might be held back, they will move on regardless. Over time they will learn the required skills, provided that they are made to practice them.

        The 7 yo in the example you give has plenty of time before he needs to be able to write about evaporation. Why does he need to be tested in any different way from the rest in the meantime — which may well hide a significant problem that his ability to understand science is not, in the long run, going to cover for.

        The Orchestra example was intended to not be limiting. Yes, those with performance anxieties can still get on in the world. But not attending to their issues will limit them compared to others, and should not be ignored merely because there are some other options open. We should be teaching our kids to give them the most options possible, not deciding when they are seven that they will never have a job where written skills are important (not, I hasten to say, that is your position — but it is a natural corollary when taken to the extreme).

        Again, as a teacher of high school, I strike problems a teacher of juniors might not have. Plenty of my students have very poor written skills. This is because over the years they have declined to put the work in to fix them — not because they lack the natural ability. (We hear so much that kids want naturally to learn — it doesn’t apply very well to recalcitrant 15 year olds.) If we start early sending them signals that they will not be required to write well, then that problem will escalate. If their earlier teachers (and parents) had insisted that they learn to spell and write properly, then they would be in a better place.

      • At the risk of a ‘back and forth’, my final post of note in regard to your follow up Chester.

        Firstly, in regard to the statement “who cares if they can’t display knowledge that they have?”. I’d say quite a few. I care, as a teacher. The parents care, the requirements of assessment and reporting care (at least here in Australia), and I would also propose that the student would care as well.

        As for students not needing to write about evaporation, our Year 2s are writing sequential explanations of scientific processes they have learned in their current unit of work. One of these is evaporation. It wasn’t a random, made-up example.

        Rather than retype, I will paste in something I commented to Greg below that responds to your fear or comment that students who need writing practice don’t and get to avoid it –
        There is no less writing being done by these students, and often the support, targeted instruction etc for these students sees them spend more time writing than some others. Not sure of your engagement in primary classrooms relative to your worry about ‘sending signals that they will not be required to write well’, but I haven’t seen that in Primary settings I have been in, and it is furthest from the messaging with my students.

        Thanks for the discussion.

    • Greg Ashman says:

      I think that the problem we have here is that UDL is suggested as a general strategy for children in all classrooms: They should be offered choices in how to complete tasks and so on. However, when I criticise this approach, people wish to characterise this as a suggestion that no accommodations at all should be made for students with *disabilities*. On Twitter, it has been repeatedly suggested that this is in violation of the law. Yet right at the start of this post I acknowledge that I am in favour of some kinds of differentiation (even if I was not focusing on disability).

      If a child has a profound physical disability or cognitive impairment that means that he or she simply cannot write then I would think it reasonable for them to show their learning in other ways e.g. by giving oral responses. However, I suspect that this is extremely rare. There are likely to be far more students who struggle with writing and who will get better at it with sustained practice and targeted instruction. If opportunities to write in science or history are substituted with other activities then these students will fall further behind their peers. And writing is an extremely important skill that should be developed by all students who are capable of developing it.

      My first level of accommodation would be as close to the original task as possible e.g. supplying a child with a writing frame to structure their response. I would exhaust these possibilities before asking them to engage in oral responses. I certainly would not leave the mode of response as a choice for students to make themselves.

      • Thanks Greg.

        It definitely isn’t about ‘choice’ in my space in this regard, it is about appropriate adjustments to allow for demonstration of knowledge and learning. At a Year 2 level, targeted instruction and practice, writing scaffolds etc. are all part of the process. They also, with the odd exception, engage in the the writing process of the task. The adjustment is made in how the student is assessed on their knowledge. The intent to not ‘penalise them twice’ for their writing limitations, otherwise, a student with difficulties as a writer would have a subject achievement profile that may more closely reflect their writing profile than their knowledge and understanding in the different subject areas.. There is no less writing being done by these students, and often the support, targeted instruction etc for these students sees them spend more time writing than some others. In a Primary School setting,with generalist classroom teaching, the management of these time allocations may well be easier than it would be in a subject specialisation in high school in this regard.

        I appreciate the clarification of your approach and thoughts.

    • My experience comes from teaching secondary, but I find some material can’t be accurately assessed through non-linguistic methods. Say, for example, the difference between a representative and direct democracy. A student could probably draw some sort of diagram, but I would still want a written explanation to fully understand the diagram and to be confident in my assessment that the student knows the difference. In which case, the student is still writing. As grade level increases, the concepts and knowledge become complex. As well, as Chester says below, I value their ability to express that knowledge as well.

      However, I can concede that some material can be represented visually, and the students can be orally tested (but with 120 students, I don’t really know how to implement that accommodation). And that your experience is with primary students, and I do not claim to understand how primary education works; I’m just grateful when my middle school students arrive with some skills and knowledge.

  7. […] my recent post on the evidence for differentiation, I noted that ‘differentiation’ is an amorphous term. I also suggested that I use […]

  8. […] Where is the evidence to support differentiation?, by Greg […]

  9. […] positive evidence from a diverse range of trials and studies yet other popular practice, such as certain forms of differentiation, have been around for a long time without generating such […]

  10. […] at the recent furor I provoked when I questioned popular forms of differentiation. The Australian Professional Standards for Teachers require teachers to differentiate! […]

  11. […] I am sceptical of the kind of differentiation represented by Tomlinson and by UDL. This is partly a logical argument: I find these forms of differentiation impractical and potentially counterproductive. However, I can also point to empirical evidence. A large U.S. study involving Tomlinson found no evidence for differentiation and a recent review of UDL found no evidence of a positive effect on educational outcomes. […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s