I am sceptical about many common approaches to ‘differentiation’ in education. This is because I worry about the practicalities of implementing these approaches, the basis on which content is differentiated and the possibility that differentiation might compound difficulties that children already have. These are genuine concerns.
However, whenever I write about this issue, I find myself criticised in strong terms. It is suggested that teachers have a legal duty to differentiate school work for students with a disability and that my scepticism about differentiation is therefore dangerous.
This is an odd argument because differentiation is a common classroom practice, whether or not students have been identified as having a disability. For instance, the video below has been produced by the Australian Institute of Teaching and School Leadership (AITSL) as an illustration of their guidelines. It involves a teacher organising a ‘guided reading’ session where the class is split into different groups on the basis of a number of factors that the teacher describes. These groups are then given different tasks to complete. Disability is not mentioned at all.
The teacher is clearly a dedicated professional who is working very hard but it is this kind of differentiation that I am sceptical about.
Interestingly, disability has become something of a hot topic in Australia in the past week. The “Gonski 2.0” proposals have brought into focus the idea of needs-based funding. As Misty Adoniou points out in The Conversation, this will be a challenge because, “Each state and sector defines disability differently. The government says it will come up with a national definition. But it does not have one yet.”
This is not just a problem for funding. If the law really does state that teachers must differentiate for students with a disability then a definition of disability might help.
This may be resolved if the Australian Labour Party succeed in their call for a Royal Commission into disability support. However, for now the situation remains confused.
It might help educators if we can distinguish between a learning disability and a learning difficulty. A document publish by the ACT government in Canberra provides some useful insights.
“Students with learning difficulties underachieve academically for a wide range of reasons, including factors such as: sensory impairment (weaknesses in vision or hearing); severe behavioural, psychological or emotional issues; English as a second language or dialect (ESL or ESD); high absenteeism; ineffective instruction; or, inadequate curricula. These students have the potential to achieve at age-appropriate levels once provided with programs that incorporate appropriate support and evidence-based instruction.
Students with learning disabilities have difficulties in specific areas of academic achievement as a result of impairment in one or more of the cognitive processes related to learning. One of the defining features of a specific learning disability is that the difficulty continues to exist, despite appropriate instruction and intervention.”
This distinction is central to my argument about whether we should accommodate or address a particular issue. If a child possesses a learning difficulty then that child needs high quality instruction to address that difficulty. If he or she struggles with writing then a writing intervention may be needed. Differentiated tasks that ask or allow such children to do less writing – a feature of differentiation approaches that are promoted in Australia such as UDL – will not address this issue and will lead, over time, to a widening gap between such children and their peers.
This does not mean that children with writing difficulties should be treated exactly the same as their peers. They may take part in withdrawal groups or make use of in-class supports. If that is what you mean by ‘differentiation’ then I have no issue with it. My criticism is specifically aimed at those forms of differentiation that lead to multiple activities taking place within one classroom and that result in a child opting out of something that they find difficult e.g. by making an audio recording rather than writing a passage.
For students with a disability as defined above then there is clearly a case for making accommodations. We wouldn’t expect a child who is blind to read from a standard textbook. Similarly, if a child possesses a neurological condition that means that he or she cannot learn to read or will never be able to read at an age-appropriate level then we would wish to make accommodations for this. Perhaps he or she could be given an audio recording or other alternative presentation of the content.
But there are genuine and serious reasons to be concerned about the kind of meta-analysis that the EEF curates. Following my last post, someone pointed me to the EEF toolkit which includes a review of collaborative learning. On face value, it looks like adding collaborative learning to whatever you are doing already will mean that your students make an extra five months of progress. How extraordinary!
When you dive into the detail, there are a few caveats. You can’t simply implement any old group work in order to obtain the advantages. You need to, “get the detail right,” and use, “structured approaches with well-designed tasks.” So you can throw out any plans you had of implementing unstructured group work with poorly designed tasks. The EEF also note that, “There is some evidence that collaboration can be supported with competition between groups, but this is not always necessary, and can lead to learners focusing on the competition rather than the learning it aims to support.”
This point is really interesting. I was surprised when I looked in to the group work evidence to find that at least some of it is based upon experiments that manipulate different systems of rewards (e.g. here). In these studies, rewards are either handed out individually or to groups and the evidence suggests that the students earning group rewards learnt the most (at least from completing assignments that seem to involve little direct teaching). This is probably why Robert Slavin emphasises the need for group goals in order for collaborative learning to be successful. it seems like the EEF are aware of this but don’t really like the finding.
I haven’t examined the papers in the EEF’s technical appendix but I note that some of them appear to be similar to Slavin’s sources. If you are interested in looking into these papers then I suggest examining the methods closely. What are the comparison groups? Is collaborative learning being tested against individually studying from a worksheet or book or is there actual teaching involved? What are the age ranges of the students involved? What are the outcomes?
Such an analysis is not the purpose of this post. According to the EEF, 5 months of progress equates to an effect size of about d=0.44 which, even if you still believe in effect sizes, is neither here nor there (see my take of effect sizes here). Moreover, the EEF state that, “The effects vary… with pooled effects between 0.09 and 0.78 but no clear explanation of what explains the variation.” Which is also neither here nor there.
And what of the absurdity of treating group work in this generic way? Are we meant to assume that we would gain five months of progress by adding group work to drama lessons? It’s hard to think of drama lessons without it. Are the findings equally valid for science and maths and reading and art?
Nevertheless, the influence of the EEF toolkit is expanding. We now have an Australian version based on the same data set. Busy school leaders will reach for it as a source of ‘evidence’.
What’s the harm? Well imagine an excellent maths teacher who instructs a whole class in an interactive way. Imagine that teacher being told to incorporate more group work because the evidence supports it. That’s the potential harm.
Group work is a default strategy for many teachers. Many forms of differentiation, such as the model used by the L3 program in New South Wales, require group work. Other teachers use it as a way of adding variety to lessons. If pushed to think about the research on group work, they may suggest it is good practice and link this to Vygotsky and social constructivism.
I tend to avoid group work in my own teaching. The only time I feel the need to create groups is when I want science students to complete practical work and resources won’t allow for them to work individually. However, my avoidance has not been due to scepticism about the effectiveness of group work. Until now, I have been persuaded by the arguments of Robert Slavin.
Briefly, Slavin surveyed the evidence on collaborative learning and found that it can be effective if two crucial conditions are in place; group goals and individual accountability. In other words, the groups need to be working towards some clearly defined objective and everyone in each group needs to be held accountable. For instance, a teacher may inform the class that one member of each group will be feeding back at the end of the session but that the teacher will decide who that is at the time. This way, everyone in the group needs to be fully briefed.
These conditions are necessary to overcome the greatest practical obstacle to group work; social loafing. Social loafing is a well-documented tendency for some group members to reduce effort and let others do the work. Most incarnations of group work don’t apply Slavin’s conditions and so this has been my main criticism.
However, I recently decided to review some of the evidence that Slavin cites in his reviews. I took a meta-analysis as my starting point.
It really hit me when I read this meta-analysis that Slavin draws a contrast between cooperative learning where there are group rewards, individualistic learning where there are rewards for reaching a certain level of performance, and competitive learning where there are rewards for performing better than other students.
This seems odd. Rewards are a feature of all conditions and the research seems to be addressing the best way of apportioning them.
One of the studies included in Slavin’s meta-analysis makes clear what this looks like. The subjects are Grade 9 science students:
“In the cooperative condition students were instructed to work together as a group, making decisions by consensus, completing the assignments together, making sure that all group members contributed their suggestions and ideas, seeking help and assistance primarily from each other rather than from the teacher, and with the teacher praising and rewarding the group as a whole. Students took the tests individually with the scores of each group’s members being totaled and averaged; each group member received the group’s average grade. A criterion- referenced evaluation system was used.”
So the students are graded on the performance of their group. For the competitive and individualistic conditions:
“In the competitive condition students were instructed to outperform all the other students in their class. A large wall-chart prominently displayed each student’s number and his or her daily and cumulative progress on all tests and assignments. The chart provided frequent and instant feedback as to the current status of each student. When the students changed teachers, the chart moved with the competitive group. The teacher verbally encouraged individual effort, made sure that students sat apart from each other, and instructed students to work on their own, asking only the teacher for help and assistance. Students were evaluated strictly on a norm-referenced basis of how their performance compared with that of their peers. The teacher praised and rewarded students who “won.”
In the individualistic condition students were instructed to work on their own, avoiding interaction with other students, seeking help and assistance from only the teacher, working at a self regulated pace, and completing as much of the assignments as possible. A criterion- referenced evaluation system was used where each student’s performance was compared to a preset criterion of excellence. The teacher praised and rewarded each student on the basis of how his or her performance compared to the preset criteria.”
Two issues stand out from this. Firstly, I cannot imagine applying any of these conditions in my routine class teaching. I tend to praise – or rather positively reinforce – effort over performance and I wouldn’t display students’ performance on a wall chart. Secondly, where is the teaching? It seems that students are working through individual assignments. Yes, they can obtain help from the teacher but there appears to be little explicit instruction. Under such conditions, I am not surprised that cooperative learning is more effective.
However, it is not that effective. The reason Slavin emphasises certain conditions for group work is that not all studies show a positive effect. In his meta-analysis, only 63% gave a significantly positive result. I suspect that there are a number of studies of this type that showed no effect and remain unpublished, so this is likely to be an optimistic figure.
I cannot locate many of the studies Slavin analyses because they are stated as ‘in press’ or are otherwise difficult to find. However, the ones that I am able to find all seem to have similar issues with the control conditions that cooperative learning is compared with. For instance, the authors of one of the studies admit that it is confounded (more than one factor was changed at a time) but suggest that the teachers involved thought the confound wouldn’t make any difference. It also seems like the cooperative learning group engaged in more retrieval practice – through the use of a game – than the control.
So I am starting to change my mind. I am not sure that the evidence for the effectiveness of group work, even if implemented under Slavin’s conditions, is sound.
I am currently rereading ‘Bad Science’ by Ben Goldacre. In the chapter on homeopathy, Goldacre sets out a number of problems with the practice. Specifically, he mentions the kinds of dilutions used by homeopaths. He asks us to picture a sphere of water with a radius equal to the distance from the Earth to the Sun. A homeopathic dilution of 30C would contain 1 molecule of the active substance in a sphere of water this volume. Homeopaths answer this critique by suggesting that water retains some kind of ‘memory’ of the active ingredient, even after it has been diluted.
Given that every water molecule has come into contact with countless substances, Goldacre finds this amusing:
“How does a water molecule know to forget every other molecule it’s seen before? How does it know to treat my bruise with its memory of arnica, rather than a memory of Isaac Asimov’s faeces? I wrote this in the newspaper once, and a homeopath complained to the Press Complaints Commission. It’s not about the dilution, he said: it’s the succussion. You have to bang the flask of water briskly ten times on a leather and horsehair surface, and that’s what makes the water remember a molecule. Because I did not mention this, he explained, I had deliberately made homeopaths sound stupid.” [Original emphasis]
There is much in this that reminds me of education’s homeopaths. We have the intense fear of ridicule. Satire is a devastating weapon against nonsense, so much so that Goldacre’s homeopath feels the need to complain about his argument to a higher authority. We have seen similar tactics in the social media education debate where offended parties threaten to launch defamation cases or complain to people’s employers.
More subtle, perhaps, is the appeal to spurious and irrelevant context. Bloggers like me are often accused of being reductive and not understanding the subtle nuances that more sophisticated and discerning pedagogues appreciate. For example, when I suggest that inquiry-based learning is ineffective for novices, there will be those who roll their eyes and shake their heads. It’s far more complicated than I appreciate, they will claim. In fact, inquiry-based learning works quite beautifully if you first bang it ten times on a leather and horsehair surface.
When I read, I hear a voice in my head. The voice is usually a pretty flat version of my own voice and it’s the same as the voice I hear when I think. Occasionally, when I receive an email from someone with a distinctive voice then I might hear their voice instead. And I sometimes hear characters’ voices when I am reading fiction.
In the past, it would never have occurred to me to mention this because I would have assumed that everyone has the same experience. But we do not. Recently a respected researcher commented in an email forum I follow that he hears no voice when he reads. I couldn’t imagine that but I did believe him so I looked into the issue and found two things; people do seem to have diverse experiences of reading ‘voices’ and yet there is little research on this phenomenon.
Let us accept that some people hear a voice when they read and some do not. If this is the case then it seems to me that there are three overlapping possibilities.
1. There are qualitative differences between people that mean that some of us hear a voice when reading and some do not.
2. There is a spectrum of experience where some are more likely to perceive a voice than others.
3. We all have fundamentally the same mental architecture but our subjective interpretation of our experience is different.
You may struggle to understand what I mean by the third possibility so here’s an attempt at trying to explain it with an analogy: Ever since I moved to Australia it has annoyed me that some Australians refer to certain people as having ‘an accent’. Everyone has an accent, I complain, especially Australians. Yet to these people, the Australian accent is so normal that they simply don’t hear it.
Perhaps some people are so familiar with the voice in their head that they don’t hear it. It’s there, but they don’t identify it as a voice because it is their own.
Whatever the case, this looks like a great phenomenon to investigate. I would speculate that people who hear a voice when reading might be relatively more inclined to accept the argument for systematic phonics teaching and those who don’t hear a voice might be relatively more inclined to be persuaded by whole language arguments about turning words directly into meaning.
Similarly, there might be a correlation between the presence of a reading voice and the way someone was taught to read. Teaching phonemic awareness and phonics might create an inner voice or it might raise awareness of a voice that would otherwise go unnoticed.
This year, researchED is partnering with the Australian College of Educators (ACE). On the 3rd July, researchED will take part in the ACE conference. If you are a member then get along to that. It will be a great day and I will be speaking along some big names.
On the Saturday 1st July, there will be a more standard researchED at Brighton Grammar School. John Hattie is the headline act and again I will be speaking.
You can book tickets here.
I would like to offer some tools that teachers, parents, journalists and others involved in education might find useful. In particular, these tools are intended to help you evaluate the kinds of claims made by presenters at conferences or in newspapers and on blogs.
1. What evidence would prove this claim wrong?
Imagine that a speaker takes to the stage and claims that a proprietary thinking routine – let’s call it ‘four hops and a ladder’ – leads to ‘deeper’ learning. The presenter shows evidence that includes lots of bar charts and demonstrates that teachers who were trained in this technique used it more often and that teachers and students alike felt motivated by it. You raise your hand to ask a question. You are aware of a study that showed no academic gains for students who used this routine versus those who did not. Ah, the presenter explain, that’s because the test that was used to assess academic gains did not assess ‘deeper’ learning. You reply, pointing out that the test included some complex transfer problems. Again, the presenter is sceptical that these really represent deeper learning. And anyway, the teachers might not have been implementing the routine properly, he notes.
Not only has the burden of proof been reversed (see below), it is hard to think of any way that we could prove this assertion wrong. The absence of evidence for something is a form of refutation but it can never be 100% conclusive. And that’s because the inductive arguments used by science can never give absolute certainty about anything. Advocates will exploit that.
It is therefore worth considering what kind of evidence really would be sufficient to disprove the proposition to the your satisfaction and to the presenter’s satisfaction. If there is a wide gulf between the two then that tells you something.
2. What are students intended to learn from this method?
Often we read about educational activities described in the most breathless terms. We may learn of maths questions so inspiring that they make it on to T-shirts or of “Aha” moments when the lights go on for particular individuals. But can you identify what students are meant to learn as a result because, often, the intended learning is not mentioned at all. All you get is the description of an activity. By definition, education has to involve learning something. So if this is not articulated then there are two possibilities. You have either been presented with an activity that has no educational benefit or the presenter has chosen not to mention what it is. Why?
3. Are you being sold motivation?
One reason that the intended learning might not be mentioned is that you’re not being sold learning at all, you’re being sold motivation. That’s fine as far as it goes but there are many fun activities in this world and many ways to pique student interest. It is all educationally useless if it doesn’t lead to more learning. If this new, motivating approach to teaching grammar leads to more and better learning of grammar then there should be evidence of that and not just evidence that it’s fun.
I am sceptical about generating what the literature terms ‘situational interest’, that is interest in the current activity and moment. I’m sure that it can aid learning but the real goal is individual interest; a long term enthusiasm for the subject. This seems to be at least partly the result of a growing sense of competence. And a sense of competence clearly relates to effective teaching practices that lead to learning.
4. Does the suggested approach sit close to the targeted skill or knowledge?
This is a tool I have thought about more recently. Imagine you want to improve a child’s reading; do you teach him or her a breathing technique or do you use a phonics based intervention. The phonics intervention directly relates to reading and the path of influence is clear; better knowledge of grapheme-phoneme correspondences will perhaps lead to improved decoding.
The chain of influence for the breathing activity is longer and more speculative. Perhaps the breathing activity will reduce anxiety. Perhaps this will then allow the child to better access his or her knowledge. Perhaps this will aid the process of reading.
If we were going to lay bets then the intervention with the shortest chain of influences would be a good choice.
5. Is the evidence a testimonial?
Education is complex, taking place in varied contexts and with many interacting components. Some people use education’s complexity to argue that the standard of evidence used by science is inappropriate and we should instead draw inferences from the kinds of sources that science largely rejects such as personal experience or anecdote.
Precisely the opposite is true. If someone is presenting you with a method to apply in your classroom then she needs to demonstrate the general effectiveness of this method. The fact that it was perceived to work in a particular context does not provide this evidence.
Scientific approaches such as experimental trials or epidemiological studies have the capacity to provide the evidence for such a general effect. The strongest approaches, such as the use of explicit instruction, can draw positive evidence from a diverse range of trials and studies yet other popular practice, such as certain forms of differentiation, have been around for a long time without generating such evidence.
6. Where lies the burden of proof?
Arguments are not always symmetrical. If someone is advocating a revolution then they bear a greater burden of proof than those who advocate for the status quo. Current practice might not be perfect but, before we jump, we should make sure we are jumping to something better.
A surprisingly large number of advocates for change simply point to flaws in the status quo. For instance, imagine someone claiming that children leave school with poor problem solving skills so we must give them more opportunity to engage in open-ended project-work. This is a weak argument.
To strengthen it, we would need some evidence to show that engaging in open-ended project-work will lead to students developing superior problem solving skills. And before we can do that, we need an understanding of what these skills are and how we can assess them. Few gurus are prepared to do this kind of ground work. It’s far easier to decry the present because you can find fault with pretty much anything.
7. Is this an argument from authority?
The Early Years Framework for Australia requires teachers to take account of children’s learning styles. However, this does not mean that the value of taking account of learning styles has been proven. Just because something is a statute or has been asserted by a figure in authority, it does not mean that it is true. In a free society, we may question such ideas.
If an argument rests solely on the authority of the person constructing it, or on an external authority, then this is not particularly persuasive. And such arguments come in many forms. Academics have an unfortunate habit of saying things like, “When you have read as much about this subject as me then you will understand.” Again, this is an argument from authority.
Challenging such an argument is tricky because it may be taken as an attack against the authority in question. So you might want to simply note the argument, factor it in to your thinking and move on.