Differentiation fails another test

Embed from Getty Images

Differentiation can mean a lot of things in teaching. If you define it widely enough, many of my own teaching practices could be described this way. However, the common meaning of the term refers to taking a mixed class of students and divide it up into different groups that have different goals and complete different tasks. I am deeply skeptical about this practice for reasons I’ve outlined before.

A new study set out to measure the impact of giving serving teachers training in within-class differentiation of this kind. As the authors, Prast et al., explain:

“With the current movement towards inclusion of children with special educational needs in general education classrooms, the range of ability and achievement levels is continuously increasing, as are the specific educational needs associated with these. Differentiation, i.e. the adaptation of instruction to students’ different educational needs, is often promoted as a solution for responding to this diversity.”

Prast et al. chose to focus on mathematics. This makes a lot of sense because it enabled the training materials to be placed in context and the effect of the intervention to be measured. They also used an interesting design. The study lasted for two years and schools were randomly assigned to one of three groups: Cohort 1 received training in year one of the study, Cohort 2 received it in year two and Cohort 3 received no training at all (but were offered it after the study ended).

What did they find?

There was a gain in student achievement in the first year of the study for students in Cohort 1 – whose teachers were the first to be trained – relative to students in the other cohorts. The gain was statistically significant but small (Prast et al. use a measure of effect size that I’m unfamiliar with but they state it is a small effect). The gain was similar for all groups of students i.e. it benefited more advanced students and less advanced students by similar amounts. However, in the second year of the study, students in Cohort 1 progressed no better than students in the other cohorts. Surprisingly, perhaps, students in Cohort 2 whose teachers received the training in the second year of the study also did no better in year two than students in the other cohorts.

The authors suggest that teachers in Cohort 1 may have been more motivated about the training than those in Cohort 2 who had to wait a year. Yet, if anything, this suggests an explanation for the positive result for Cohort 1 in the first year. It may have been due to an expectation effect – these teachers expected the intervention to make a difference and probably communicated this enthusiasm to their students. Such effects are common in education research because we cannot easily blind trials – subjects know whether they are in the intervention or not. Teachers were also likely to have spent more time thinking about their maths teaching during that year. This would explain why the effect washed out by the second year.

This is not the first study to find a null or negative result for differentiation. I have commented on other studies before. The evidence supporting differentiation is highly elusive but this has not stopped it becoming an article of faith among education bureaucrats and researchers. When I write about differentiation, I can expect strong, negative responses. I think that, as Prast et al. suggest, differentiation has become conflated with views about inclusion and they interpret criticism as an attack on children with special educational needs. If that’s true, we should find some evidence showing that within-class differentiation helps children with these needs to learn. Where is that?


10 thoughts on “Differentiation fails another test

  1. That expectation effect seems to be ignored and/or downplayed in so much education research, and yet it’s crucial. This is why I think some of the most reliable education research deals not so much with new interventions but longitudinal comparisons of two teachers/schools/systems who/which have already decided on, and are familiar with and committed to, different approaches. But of course it’s very hard to adjust for other factors in that case as well.

  2. Hi Greg
    I think in common parlance (at least in secondary) differentiation doesn’t even have different goals. I think most teachers view it as – what’s some things to do with the students who get this first time, and how can I scaffold it more for students struggling.
    I know there are people that push for more differentiation than this (amazingly it often seems to be the same people that criticise selective schools or streaming which I see as a bit of a contradiction) and it doesn’t surprise me that these perform badly. I can think of ‘exemplars’ of 10 hour projects of colonising Mars where the advanced group are expected to detail and demonstrate a possible terraforming process and the struggling group make a plasticine base (taking about 10 minutes of research and 50 minutes to construct).
    Many of the arguments you’ve been having recently over disability we would probably call ‘accessibility’ rather than differentiation and personally I listen to what our ILP team says is required and follow that to the letter.

      1. Individual learning plan. Basically our inclusion support team write ILPs for all students for all students in all subjects with diagnosed disabilities etc. They work with the teachers and parents and have found them to be quite reasonable with their adjustments

  3. I’ve not heard the term ‘expectation effect’ before–does it differ in any significant way from the Hawthorne Effect?

  4. Without knowing more about the “professional development” these teachers received, this sounds a lot more like tracking than differentiation, which would explain why the only significant effect was based on expectations (which are known to raise student achievement). For my part, I’m skeptical of this study’s use of ability grouping. Ability grouping is most effective when the groups are dynamic and flexible, but few teachers practice it that way. If teachers do use ability groups, I hope they’re also re-teaching, scaffolding, and allowing students to express their understanding in different ways — these are all forms of differentiation that help all students meet common goals by different paths. Absent that, it’s no wonder the effects weren’t significant.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.