This is the homepage of Greg Ashman, a teacher living and working in Australia. Nothing that I write or that I link to necessarily reflects the view of my school.
I have written for Spiked magazine
I have written for The Conversation:
Some of my writing is also on the researchED website workingoutwhatworks.com
I used to write articles for the the TES. These now appear to have been paywalled. I will probably make them available on my blog at some point. If you have access then you can find them here:
Create waves of learning, Master the mysterious art of explanation, Taking a critical look at praise, Behaviour, Great Scott! Let’s push the brain to its limits, The science fiction that doing is best, Make them sit up and take notice, For great rewards, sweat the small stuff, Where the grass is browner, Stand-out teaching – minus differentiation
But there are genuine and serious reasons to be concerned about the kind of meta-analysis that the EEF curates. Following my last post, someone pointed me to the EEF toolkit which includes a review of collaborative learning. On face value, it looks like adding collaborative learning to whatever you are doing already will mean that your students make an extra five months of progress. How extraordinary!
When you dive into the detail, there are a few caveats. You can’t simply implement any old group work in order to obtain the advantages. You need to, “get the detail right,” and use, “structured approaches with well-designed tasks.” So you can throw out any plans you had of implementing unstructured group work with poorly designed tasks. The EEF also note that, “There is some evidence that collaboration can be supported with competition between groups, but this is not always necessary, and can lead to learners focusing on the competition rather than the learning it aims to support.”
This point is really interesting. I was surprised when I looked in to the group work evidence to find that at least some of it is based upon experiments that manipulate different systems of rewards (e.g. here). In these studies, rewards are either handed out individually or to groups and the evidence suggests that the students earning group rewards learnt the most (at least from completing assignments that seem to involve little direct teaching). This is probably why Robert Slavin emphasises the need for group goals in order for collaborative learning to be successful. it seems like the EEF are aware of this but don’t really like the finding.
I haven’t examined the papers in the EEF’s technical appendix but I note that some of them appear to be similar to Slavin’s sources. If you are interested in looking into these papers then I suggest examining the methods closely. What are the comparison groups? Is collaborative learning being tested against individually studying from a worksheet or book or is there actual teaching involved? What are the age ranges of the students involved? What are the outcomes?
Such an analysis is not the purpose of this post. According to the EEF, 5 months of progress equates to an effect size of about d=0.44 which, even if you still believe in effect sizes, is neither here nor there (see my take of effect sizes here). Moreover, the EEF state that, “The effects vary… with pooled effects between 0.09 and 0.78 but no clear explanation of what explains the variation.” Which is also neither here nor there.
And what of the absurdity of treating group work in this generic way? Are we meant to assume that we would gain five months of progress by adding group work to drama lessons? It’s hard to think of drama lessons without it. Are the findings equally valid for science and maths and reading and art?
Nevertheless, the influence of the EEF toolkit is expanding. We now have an Australian version based on the same data set. Busy school leaders will reach for it as a source of ‘evidence’.
What’s the harm? Well imagine an excellent maths teacher who instructs a whole class in an interactive way. Imagine that teacher being told to incorporate more group work because the evidence supports it. That’s the potential harm.
Group work is a default strategy for many teachers. Many forms of differentiation, such as the model used by the L3 program in New South Wales, require group work. Other teachers use it as a way of adding variety to lessons. If pushed to think about the research on group work, they may suggest it is good practice and link this to Vygotsky and social constructivism.
I tend to avoid group work in my own teaching. The only time I feel the need to create groups is when I want science students to complete practical work and resources won’t allow for them to work individually. However, my avoidance has not been due to scepticism about the effectiveness of group work. Until now, I have been persuaded by the arguments of Robert Slavin.
Briefly, Slavin surveyed the evidence on collaborative learning and found that it can be effective if two crucial conditions are in place; group goals and individual accountability. In other words, the groups need to be working towards some clearly defined objective and everyone in each group needs to be held accountable. For instance, a teacher may inform the class that one member of each group will be feeding back at the end of the session but that the teacher will decide who that is at the time. This way, everyone in the group needs to be fully briefed.
These conditions are necessary to overcome the greatest practical obstacle to group work; social loafing. Social loafing is a well-documented tendency for some group members to reduce effort and let others do the work. Most incarnations of group work don’t apply Slavin’s conditions and so this has been my main criticism.
However, I recently decided to review some of the evidence that Slavin cites in his reviews. I took a meta-analysis as my starting point.
It really hit me when I read this meta-analysis that Slavin draws a contrast between cooperative learning where there are group rewards, individualistic learning where there are rewards for reaching a certain level of performance, and competitive learning where there are rewards for performing better than other students.
This seems odd. Rewards are a feature of all conditions and the research seems to be addressing the best way of apportioning them.
One of the studies included in Slavin’s meta-analysis makes clear what this looks like. The subjects are Grade 9 science students:
“In the cooperative condition students were instructed to work together as a group, making decisions by consensus, completing the assignments together, making sure that all group members contributed their suggestions and ideas, seeking help and assistance primarily from each other rather than from the teacher, and with the teacher praising and rewarding the group as a whole. Students took the tests individually with the scores of each group’s members being totaled and averaged; each group member received the group’s average grade. A criterion- referenced evaluation system was used.”
So the students are graded on the performance of their group. For the competitive and individualistic conditions:
“In the competitive condition students were instructed to outperform all the other students in their class. A large wall-chart prominently displayed each student’s number and his or her daily and cumulative progress on all tests and assignments. The chart provided frequent and instant feedback as to the current status of each student. When the students changed teachers, the chart moved with the competitive group. The teacher verbally encouraged individual effort, made sure that students sat apart from each other, and instructed students to work on their own, asking only the teacher for help and assistance. Students were evaluated strictly on a norm-referenced basis of how their performance compared with that of their peers. The teacher praised and rewarded students who “won.”
In the individualistic condition students were instructed to work on their own, avoiding interaction with other students, seeking help and assistance from only the teacher, working at a self regulated pace, and completing as much of the assignments as possible. A criterion- referenced evaluation system was used where each student’s performance was compared to a preset criterion of excellence. The teacher praised and rewarded each student on the basis of how his or her performance compared to the preset criteria.”
Two issues stand out from this. Firstly, I cannot imagine applying any of these conditions in my routine class teaching. I tend to praise – or rather positively reinforce – effort over performance and I wouldn’t display students’ performance on a wall chart. Secondly, where is the teaching? It seems that students are working through individual assignments. Yes, they can obtain help from the teacher but there appears to be little explicit instruction. Under such conditions, I am not surprised that cooperative learning is more effective.
However, it is not that effective. The reason Slavin emphasises certain conditions for group work is that not all studies show a positive effect. In his meta-analysis, only 63% gave a significantly positive result. I suspect that there are a number of studies of this type that showed no effect and remain unpublished, so this is likely to be an optimistic figure.
I cannot locate many of the studies Slavin analyses because they are stated as ‘in press’ or are otherwise difficult to find. However, the ones that I am able to find all seem to have similar issues with the control conditions that cooperative learning is compared with. For instance, the authors of one of the studies admit that it is confounded (more than one factor was changed at a time) but suggest that the teachers involved thought the confound wouldn’t make any difference. It also seems like the cooperative learning group engaged in more retrieval practice – through the use of a game – than the control.
So I am starting to change my mind. I am not sure that the evidence for the effectiveness of group work, even if implemented under Slavin’s conditions, is sound.
I am currently rereading ‘Bad Science’ by Ben Goldacre. In the chapter on homeopathy, Goldacre sets out a number of problems with the practice. Specifically, he mentions the kinds of dilutions used by homeopaths. He asks us to picture a sphere of water with a radius equal to the distance from the Earth to the Sun. A homeopathic dilution of 30C would contain 1 molecule of the active substance in a sphere of water this volume. Homeopaths answer this critique by suggesting that water retains some kind of ‘memory’ of the active ingredient, even after it has been diluted.
Given that every water molecule has come into contact with countless substances, Goldacre finds this amusing:
“How does a water molecule know to forget every other molecule it’s seen before? How does it know to treat my bruise with its memory of arnica, rather than a memory of Isaac Asimov’s faeces? I wrote this in the newspaper once, and a homeopath complained to the Press Complaints Commission. It’s not about the dilution, he said: it’s the succussion. You have to bang the flask of water briskly ten times on a leather and horsehair surface, and that’s what makes the water remember a molecule. Because I did not mention this, he explained, I had deliberately made homeopaths sound stupid.” [Original emphasis]
There is much in this that reminds me of education’s homeopaths. We have the intense fear of ridicule. Satire is a devastating weapon against nonsense, so much so that Goldacre’s homeopath feels the need to complain about his argument to a higher authority. We have seen similar tactics in the social media education debate where offended parties threaten to launch defamation cases or complain to people’s employers.
More subtle, perhaps, is the appeal to spurious and irrelevant context. Bloggers like me are often accused of being reductive and not understanding the subtle nuances that more sophisticated and discerning pedagogues appreciate. For example, when I suggest that inquiry-based learning is ineffective for novices, there will be those who roll their eyes and shake their heads. It’s far more complicated than I appreciate, they will claim. In fact, inquiry-based learning works quite beautifully if you first bang it ten times on a leather and horsehair surface.
When I read, I hear a voice in my head. The voice is usually a pretty flat version of my own voice and it’s the same as the voice I hear when I think. Occasionally, when I receive an email from someone with a distinctive voice then I might hear their voice instead. And I sometimes hear characters’ voices when I am reading fiction.
In the past, it would never have occurred to me to mention this because I would have assumed that everyone has the same experience. But we do not. Recently a respected researcher commented in an email forum I follow that he hears no voice when he reads. I couldn’t imagine that but I did believe him so I looked into the issue and found two things; people do seem to have diverse experiences of reading ‘voices’ and yet there is little research on this phenomenon.
Let us accept that some people hear a voice when they read and some do not. If this is the case then it seems to me that there are three overlapping possibilities.
1. There are qualitative differences between people that mean that some of us hear a voice when reading and some do not.
2. There is a spectrum of experience where some are more likely to perceive a voice than others.
3. We all have fundamentally the same mental architecture but our subjective interpretation of our experience is different.
You may struggle to understand what I mean by the third possibility so here’s an attempt at trying to explain it with an analogy: Ever since I moved to Australia it has annoyed me that some Australians refer to certain people as having ‘an accent’. Everyone has an accent, I complain, especially Australians. Yet to these people, the Australian accent is so normal that they simply don’t hear it.
Perhaps some people are so familiar with the voice in their head that they don’t hear it. It’s there, but they don’t identify it as a voice because it is their own.
Whatever the case, this looks like a great phenomenon to investigate. I would speculate that people who hear a voice when reading might be relatively more inclined to accept the argument for systematic phonics teaching and those who don’t hear a voice might be relatively more inclined to be persuaded by whole language arguments about turning words directly into meaning.
Similarly, there might be a correlation between the presence of a reading voice and the way someone was taught to read. Teaching phonemic awareness and phonics might create an inner voice or it might raise awareness of a voice that would otherwise go unnoticed.
This year, researchED is partnering with the Australian College of Educators (ACE). On the 3rd July, researchED will take part in the ACE conference. If you are a member then get along to that. It will be a great day and I will be speaking along some big names.
On the Saturday 1st July, there will be a more standard researchED at Brighton Grammar School. John Hattie is the headline act and again I will be speaking.
You can book tickets here.
I would like to offer some tools that teachers, parents, journalists and others involved in education might find useful. In particular, these tools are intended to help you evaluate the kinds of claims made by presenters at conferences or in newspapers and on blogs.
1. What evidence would prove this claim wrong?
Imagine that a speaker takes to the stage and claims that a proprietary thinking routine – let’s call it ‘four hops and a ladder’ – leads to ‘deeper’ learning. The presenter shows evidence that includes lots of bar charts and demonstrates that teachers who were trained in this technique used it more often and that teachers and students alike felt motivated by it. You raise your hand to ask a question. You are aware of a study that showed no academic gains for students who used this routine versus those who did not. Ah, the presenter explain, that’s because the test that was used to assess academic gains did not assess ‘deeper’ learning. You reply, pointing out that the test included some complex transfer problems. Again, the presenter is sceptical that these really represent deeper learning. And anyway, the teachers might not have been implementing the routine properly, he notes.
Not only has the burden of proof been reversed (see below), it is hard to think of any way that we could prove this assertion wrong. The absence of evidence for something is a form of refutation but it can never be 100% conclusive. And that’s because the inductive arguments used by science can never give absolute certainty about anything. Advocates will exploit that.
It is therefore worth considering what kind of evidence really would be sufficient to disprove the proposition to the your satisfaction and to the presenter’s satisfaction. If there is a wide gulf between the two then that tells you something.
2. What are students intended to learn from this method?
Often we read about educational activities described in the most breathless terms. We may learn of maths questions so inspiring that they make it on to T-shirts or of “Aha” moments when the lights go on for particular individuals. But can you identify what students are meant to learn as a result because, often, the intended learning is not mentioned at all. All you get is the description of an activity. By definition, education has to involve learning something. So if this is not articulated then there are two possibilities. You have either been presented with an activity that has no educational benefit or the presenter has chosen not to mention what it is. Why?
3. Are you being sold motivation?
One reason that the intended learning might not be mentioned is that you’re not being sold learning at all, you’re being sold motivation. That’s fine as far as it goes but there are many fun activities in this world and many ways to pique student interest. It is all educationally useless if it doesn’t lead to more learning. If this new, motivating approach to teaching grammar leads to more and better learning of grammar then there should be evidence of that and not just evidence that it’s fun.
I am sceptical about generating what the literature terms ‘situational interest’, that is interest in the current activity and moment. I’m sure that it can aid learning but the real goal is individual interest; a long term enthusiasm for the subject. This seems to be at least partly the result of a growing sense of competence. And a sense of competence clearly relates to effective teaching practices that lead to learning.
4. Does the suggested approach sit close to the targeted skill or knowledge?
This is a tool I have thought about more recently. Imagine you want to improve a child’s reading; do you teach him or her a breathing technique or do you use a phonics based intervention. The phonics intervention directly relates to reading and the path of influence is clear; better knowledge of grapheme-phoneme correspondences will perhaps lead to improved decoding.
The chain of influence for the breathing activity is longer and more speculative. Perhaps the breathing activity will reduce anxiety. Perhaps this will then allow the child to better access his or her knowledge. Perhaps this will aid the process of reading.
If we were going to lay bets then the intervention with the shortest chain of influences would be a good choice.
5. Is the evidence a testimonial?
Education is complex, taking place in varied contexts and with many interacting components. Some people use education’s complexity to argue that the standard of evidence used by science is inappropriate and we should instead draw inferences from the kinds of sources that science largely rejects such as personal experience or anecdote.
Precisely the opposite is true. If someone is presenting you with a method to apply in your classroom then she needs to demonstrate the general effectiveness of this method. The fact that it was perceived to work in a particular context does not provide this evidence.
Scientific approaches such as experimental trials or epidemiological studies have the capacity to provide the evidence for such a general effect. The strongest approaches, such as the use of explicit instruction, can draw positive evidence from a diverse range of trials and studies yet other popular practice, such as certain forms of differentiation, have been around for a long time without generating such evidence.
6. Where lies the burden of proof?
Arguments are not always symmetrical. If someone is advocating a revolution then they bear a greater burden of proof than those who advocate for the status quo. Current practice might not be perfect but, before we jump, we should make sure we are jumping to something better.
A surprisingly large number of advocates for change simply point to flaws in the status quo. For instance, imagine someone claiming that children leave school with poor problem solving skills so we must give them more opportunity to engage in open-ended project-work. This is a weak argument.
To strengthen it, we would need some evidence to show that engaging in open-ended project-work will lead to students developing superior problem solving skills. And before we can do that, we need an understanding of what these skills are and how we can assess them. Few gurus are prepared to do this kind of ground work. It’s far easier to decry the present because you can find fault with pretty much anything.
7. Is this an argument from authority?
The Early Years Framework for Australia requires teachers to take account of children’s learning styles. However, this does not mean that the value of taking account of learning styles has been proven. Just because something is a statute or has been asserted by a figure in authority, it does not mean that it is true. In a free society, we may question such ideas.
If an argument rests solely on the authority of the person constructing it, or on an external authority, then this is not particularly persuasive. And such arguments come in many forms. Academics have an unfortunate habit of saying things like, “When you have read as much about this subject as me then you will understand.” Again, this is an argument from authority.
Challenging such an argument is tricky because it may be taken as an attack against the authority in question. So you might want to simply note the argument, factor it in to your thinking and move on.
Australian researcher Ben Jensen heads an organisation called, ‘Learning First’. Jacqueline Magee of Learning First recently published a blog post drawing attention to a Learning First report and its call for greater subject specialisation by Australian primary school teachers.
This is clearly a sound idea.
To some people, the case for non-specialist primary teaching may be strong. There are two main reasons for sticking with a generalist approach, the most compelling of which is the idea that young children need a key point of contact at school who comes to know each child very well. I agree that this is important. However, a child can still have most of his or her lessons with a particular teacher in a primary school that adopts some specialisation. My own experience suggests that children are more adaptable than we might presume, although I can imagine that it will be a challenge for some students.
The other argument against specialisation is that we no longer need to focus on academic knowledge. By targeting an expanded range of ‘literacies’ and generic skills such as problem solving, critical thinking, collaboration and creativity, there is no need for specialist knowledge. Indeed, through project work, students may engage with a wide range of knowledge that is not predictable in advance by the teacher.
I am of the view that it is flawed to imagine that such general skills exist and can be taught in this way. In order to think critically, we need something to think critically about. Dan Willingham makes the point that small children can think critically about areas of knowledge that they know about and trained scientists can fail to think critically about areas of knowledge they know little about. Knowledge is what you think with. And there is little that transfers from solving the problem of how to finish a story to solving a mathematics problem.
Teachers all over the world are now part of a growing movement that recognises that traditional subject disciplines are not arbitrary. These subjects exist because they define powerful domains of knowledge and important ways of knowing. If we want children to be able to demonstrate the qualities that we value such as critical thinking or creativity in a sophisticated way then they need to have gained this important knowledge and grappled with these big ideas.
In some cases, primary school teachers may be able to adopt a specific interest in order to develop a specialism. For instance, a teacher may become the lead on history and read around the history units in detail so that his or her knowledge is greater than that which is intended to be taught to the students.
However, in some specialisms we might encounter a problem.
Science and mathematics are highly prized specialisms and we hardly have enough graduates to specialise in these subjects in secondary school. Some primary teachers are definitely capable of being experts in these areas but they will represent more of a challenge for others. Ultimately, we need to find ways to encourage more people with these backgrounds into teaching.
That’s going to be tough.
Update: a number of people on Twitter and in the comments have drawn attention to this RCT from Texas that seems to show a negative impact of specialisation.