A voice in your head?

When I read, I hear a voice in my head. The voice is usually a pretty flat version of my own voice and it’s the same as the voice I hear when I think. Occasionally, when I receive an email from someone with a distinctive voice then I might hear their voice instead. And I sometimes hear characters’ voices when I am reading fiction.

In the past, it would never have occurred to me to mention this because I would have assumed that everyone has the same experience. But we do not. Recently a respected researcher commented in an email forum I follow that he hears no voice when he reads. I couldn’t imagine that but I did believe him so I looked into the issue and found two things; people do seem to have diverse experiences of reading ‘voices’ and yet there is little research on this phenomenon.

Let us accept that some people hear a voice when they read and some do not. If this is the case then it seems to me that there are three overlapping possibilities.

1. There are qualitative differences between people that mean that some of us hear a voice when reading and some do not.

2. There is a spectrum of experience where some are more likely to perceive a voice than others.

3. We all have fundamentally the same mental architecture but our subjective interpretation of our experience is different.

You may struggle to understand what I mean by the third possibility so here’s an attempt at trying to explain it with an analogy: Ever since I moved to Australia it has annoyed me that some Australians refer to certain people as having ‘an accent’. Everyone has an accent, I complain, especially Australians. Yet to these people, the Australian accent is so normal that they simply don’t hear it.

Perhaps some people are so familiar with the voice in their head that they don’t hear it. It’s there, but they don’t identify it as a voice because it is their own.

Whatever the case, this looks like a great phenomenon to investigate. I would speculate that people who hear a voice when reading might be relatively more inclined to accept the argument for systematic phonics teaching and those who don’t hear a voice might be relatively more inclined to be persuaded by whole language arguments about turning words directly into meaning.

Similarly, there might be a correlation between the presence of a reading voice and the way someone was taught to read. Teaching phonemic awareness and phonics might create an inner voice or it might raise awareness of a voice that would otherwise go unnoticed.

See you at researchED Melbourne, 1st July 2017 #aussieED

This year, researchED is partnering with the Australian College of Educators (ACE). On the 3rd July, researchED will take part in the ACE conference. If you are a member then get along to that. It will be a great day and I will be speaking along some big names.

On the Saturday 1st July, there will be a more standard researchED at Brighton Grammar School. John Hattie is the headline act and again I will be speaking.

You can book tickets here

7 tools for thinking about education

I would like to offer some tools that teachers, parents, journalists and others involved in education might find useful. In particular, these tools are intended to help you evaluate the kinds of claims made by presenters at conferences or in newspapers and on blogs.

1. What evidence would prove this claim wrong? 

Imagine that a speaker takes to the stage and claims that a proprietary thinking routine – let’s call it ‘four hops and a ladder’ – leads to ‘deeper’ learning. The presenter shows evidence that includes lots of bar charts and demonstrates that teachers who were trained in this technique used it more often and that teachers and students alike felt motivated by it. You raise your hand to ask a question. You are aware of a study that showed no academic gains for students who used this routine versus those who did not. Ah, the presenter explain, that’s because the test that was used to assess academic gains did not assess ‘deeper’ learning. You reply, pointing out that the test included some complex transfer problems. Again, the presenter is sceptical that these really represent deeper learning. And anyway, the teachers might not have been implementing the routine properly, he notes.

Not only has the burden of proof been reversed (see below), it is hard to think of any way that we could prove this assertion wrong. The absence of evidence for something is a form of refutation but it can never be 100% conclusive. And that’s because the inductive arguments used by science can never give absolute certainty about anything. Advocates will exploit that.

It is therefore worth considering what kind of evidence really would be sufficient to disprove the proposition to the your satisfaction and to the presenter’s satisfaction. If there is a wide gulf between the two then that tells you something.

2. What are students intended to learn from this method?

Often we read about educational activities described in the most breathless terms. We may learn of maths questions so inspiring that they make it on to T-shirts or of “Aha” moments when the lights go on for particular individuals. But can you identify what students are meant to learn as a result because, often, the intended learning is not mentioned at all. All you get is the description of an activity. By definition, education has to involve learning something. So if this is not articulated then there are two possibilities. You have either been presented with an activity that has no educational benefit or the presenter has chosen not to mention what it is. Why?

3. Are you being sold motivation?

One reason that the intended learning might not be mentioned is that you’re not being sold learning at all, you’re being sold motivation. That’s fine as far as it goes but there are many fun activities in this world and many ways to pique student interest. It is all educationally useless if it doesn’t lead to more learning. If this new, motivating approach to teaching grammar leads to more and better learning of grammar then there should be evidence of that and not just evidence that it’s fun.

I am sceptical about generating what the literature terms ‘situational interest’, that is interest in the current activity and moment. I’m sure that it can aid learning but the real goal is individual interest; a long term enthusiasm for the subject. This seems to be at least partly the result of a growing sense of competence. And a sense of competence clearly relates to effective teaching practices that lead to learning.

4. Does the suggested approach sit close to the targeted skill or knowledge?

This is a tool I have thought about more recently. Imagine you want to improve a child’s reading; do you teach him or her a breathing technique or do you use a phonics based intervention. The phonics intervention directly relates to reading and the path of influence is clear; better knowledge of grapheme-phoneme correspondences will perhaps lead to improved decoding.

The chain of influence for the breathing activity is longer and more speculative. Perhaps the breathing activity will reduce anxiety. Perhaps this will then allow the child to better access his or her knowledge. Perhaps this will aid the process of reading. 

If we were going to lay bets then the intervention with the shortest chain of influences would be a good choice.

5. Is the evidence a testimonial?

Education is complex, taking place in varied contexts and with many interacting components. Some people use education’s complexity to argue that the standard of evidence used by science is inappropriate and we should instead draw inferences from the kinds of sources that science largely rejects such as personal experience or anecdote.

Precisely the opposite is true. If someone is presenting you with a method to apply in your classroom then she needs to demonstrate the general effectiveness of this method. The fact that it was perceived to work in a particular context does not provide this evidence.

Scientific approaches such as experimental trials or epidemiological studies have the capacity to provide the evidence for such a general effect. The strongest approaches, such as the use of explicit instruction, can draw positive evidence from a diverse range of trials and studies yet other popular practice, such as certain forms of differentiation, have been around for a long time without generating such evidence. 

6. Where lies the burden of proof?

Arguments are not always symmetrical. If someone is advocating a revolution then they bear a greater burden of proof than those who advocate for the status quo. Current practice might not be perfect but, before we jump, we should make sure we are jumping to something better.

A surprisingly large number of advocates for change simply point to flaws in the status quo. For instance, imagine someone claiming that children leave school with poor problem solving skills so we must give them more opportunity to engage in open-ended project-work. This is a weak argument. 

To strengthen it, we would need some evidence to show that engaging in open-ended project-work will lead to students developing superior problem solving skills. And before we can do that, we need an understanding of what these skills are and how we can assess them. Few gurus are prepared to do this kind of ground work. It’s far easier to decry the present because you can find fault with pretty much anything.

7. Is this an argument from authority?

The Early Years Framework for Australia requires teachers to take account of children’s learning styles. However, this does not mean that the value of taking account of learning styles has been proven. Just because something is a statute or has been asserted by a figure in authority, it does not mean that it is true. In a free society, we may question such ideas.

If an argument rests solely on the authority of the person constructing it, or on an external authority, then this is not particularly persuasive. And such arguments come in many forms. Academics have an unfortunate habit of saying things like, “When you have read as much about this subject as me then you will understand.” Again, this is an argument from authority.

Challenging such an argument is tricky because it may be taken as an attack against the authority in question. So you might want to simply note the argument, factor it in to your thinking and move on.

Should we have more specialist teachers in primary schools? 

Australian researcher Ben Jensen heads an organisation called, ‘Learning First’. Jacqueline Magee of Learning First recently published a blog post drawing attention to a Learning First report and its call for greater subject specialisation by Australian primary school teachers.

This is clearly a sound idea.

To some people, the case for non-specialist primary teaching may be strong. There are two main reasons for sticking with a generalist approach, the most compelling of which is the idea that young children need a key point of contact at school who comes to know each child very well. I agree that this is important. However, a child can still have most of his or her lessons with a particular teacher in a primary school that adopts some specialisation. My own experience suggests that children are more adaptable than we might presume, although I can imagine that it will be a challenge for some students.

The other argument against specialisation is that we no longer need to focus on academic knowledge. By targeting an expanded range of ‘literacies’ and generic skills such as problem solving, critical thinking, collaboration and creativity, there is no need for specialist knowledge. Indeed, through project work, students may engage with a wide range of knowledge that is not predictable in advance by the teacher.

I am of the view that it is flawed to imagine that such general skills exist and can be taught in this way. In order to think critically, we need something to think critically about. Dan Willingham makes the point that small children can think critically about areas of knowledge that they know about and trained scientists can fail to think critically about areas of knowledge they know little about. Knowledge is what you think with. And there is little that transfers from solving the problem of how to finish a story to solving a mathematics problem.

Teachers all over the world are now part of a growing movement that recognises that traditional subject disciplines are not arbitrary. These subjects exist because they define powerful domains of knowledge and important ways of knowing. If we want children to be able to demonstrate the qualities that we value such as critical thinking or creativity in a sophisticated way then they need to have gained this important knowledge and grappled with these big ideas.

In some cases, primary school teachers may be able to adopt a specific interest in order to develop a specialism. For instance, a teacher may become the lead on history and read around the history units in detail so that his or her knowledge is greater than that which is intended to be taught to the students.

However, in some specialisms we might encounter a problem.

Science and mathematics are highly prized specialisms and we hardly have enough graduates to specialise in these subjects in secondary school. Some primary teachers are definitely capable of being experts in these areas but they will represent more of a challenge for others. Ultimately, we need to find ways to encourage more people with these backgrounds into teaching.

That’s going to be tough.

Update: a number of people on Twitter and in the comments have drawn attention to this RCT from Texas that seems to show a negative impact of specialisation. 

Differentiation’s slippery slope


I have recently discovered that some people on Twitter don’t want to engage in my arguments but instead simply wish to repeat points over my head as they play to the gallery. One of these points is particularly interesting because it is a subtle manipulation of my views on differentiation; a bait and switch.

In my recent post on the evidence for differentiation, I noted that ‘differentiation’ is an amorphous term. I also suggested that I use certain forms of it in my own teaching. I criticised specific kinds of differentiation typified by Universal Design for Learning (UDL) and the Differentiated Instruction techniques of Carol Ann Tomlinson, both of which have been promoted by academics in Australia. As Tomlinson explains, in her approach:

“Teachers can differentiate at least four classroom elements based on student readiness, interest, or learning profile: (1) content—what the student needs to learn or how the student will get access to the information; (2) process—activities in which the student engages in order to make sense of or master the content; (3) products—culminating projects that ask the student to rehearse, apply, and extend what he or she has learned in a unit; and (4) learning environment—the way the classroom works and feels.”

UDL is similar and stresses the element of student choice of classroom tasks. In my post, I raised two quite reasonable concerns about such forms of differentiation. Firstly, if a child struggles with writing then a teacher applying these principles may give that child less writing to complete or allow that child to choose to complete less writing. If this happens over time across a number of subjects then this child is likely to fall further behind his or her peers in writing. The same could be said of any other academic skill. Secondly, orchestrating the kind of differentiation where students complete multiple activities in a single class is inefficient for teachers to plan and deliver.

In my post, I noted the lack of empirical evidence to support the idea that this form of differentiation leads to better educational outcomes.

Unfortunately, some people chose to interpret my comments entirely through the lens of disability. Let me be explicit: Tomlinson and the UDL developers are promoting their forms of differentiation as a general strategy and not one to be reserved for students with a disability.

Once this strawman had been erected, a number of people repeatedly and persistently claimed that it is illegal to refuse to make accommodations for students with a disability under the Australian Disability Standards for Education (DSE).

I have read the standards and I do not think that they require any teacher to make use of the forms of differentiation typified by UDL or Tomlinson. Instead, they require teachers to make reasonable accommodations for students with disabilities. Such accommodations are defined alongside the limits of what is considered reasonable.

Let me be clear: If a child has a cognitive or physical impairment that means that he or she cannot learn to write then it would be cruel and absurd to insist on that child writing. However, I suspect that this is relatively rare. More often, I would expect to encounter a child who has difficulty in learning to write. If I were to make accommodations for such a child then it might be by providing additional help with writing, providing a writing frame or offering a similar support. UDL is clearly flawed in my view because I would not give that child a choice to opt out of writing.

Now let us for the sake of argument assume that I am wrong and that the DSE really does require all Australian teachers to use a UDL-style approach to differentiation. If that were the case then I would argue that the law was wrong. This is a perfectly respectable position to take in a free society. For instance, I personally believe that same-sex couples should be able to marry, something that is not currently permitted by law in Australia. I am not going to be convinced that I am wrong by someone repeatedly stating that the law does not allow it.

Please feel free to engage me on the substance of my argument. Some already have engaged constructively. I am often wrong and I may well be wrong about this. However, I will no longer engage with those who are seeking to frame this discussion in a way that misrepresents my position.

Cognitive load of rubbish?

I recently wrote a post on how knowledge of cognitive load theory (CLT) has changed my practice as a teacher. It was shared quite widely. I tend to attract critics of CLT and I think this is healthy. It is a relatively new theory and a reasonable and measured response is to take its finding provisionally. My own view of CLT is that the empirical studies are pretty robust but the theory is still under development.

When one fairly high profile person shared my post, Dr Sandra Leaton Gray responded on Twitter with some pointed criticism of CLT:

These criticisms immediately struck me as odd for two reasons. Firstly, CLT originated in Australia before a number of researchers from the Netherlands got hold of it. I wasn’t aware of many U.S. sources on the theory and had been under the impression that the U.S. was largely uninterested in it. And I seemed to recall John Sweller describing studies he conducted with Year 9 students, contradicting the suggestion that the sources are biased towards HE (higher education) and therefore the age group is too old.

I did a search for the original Sweller and Cooper studies on the worked example effect that sit at the root of CLT and found this paper. The subjects of this study were Australian (not U.S.) students from Year 9, Year 11 and university. Only Year 9 students were used for the critical tests of worked examples versus problem solving. The authors even discuss the fact that university students possess too much expertise in solving the required algebra problems for them to act as good test subjects.

The self-citation claim seems accurate but perhaps a little unfair. When there are only a small number of people working on a theory then who else should researchers’ cite? I think this was one of the reasons that Sweller and colleagues were pleased when CLT became popular among academics in The Netherlands.

Another issue that Leaton Gray raised was that the studies that form the basis of CLT are too old:

Again, this struck me as odd. Why does the age of the studies matter? Have children’s brain’s changed since the 1980s? That seems a short timescale for evolution.

And I am doing research right now into cognitive load theory with Australian school students. There are new CLT papers appearing in my various RSS feeds every day. If we look at the papers that form key points in the development of the theory then there is trail that leads right from the 1980s up to the present day. A quick search of Google Scholar with return relevant papers from every intermediate point e.g. this paper on the expertise reversal effect from 2003 or this paper from 2010 that demonstrates a worked example effect for an annotated Shakespeare play (participants: Year 10 students from Sydney).

Leaton Gray also suggested that neuroscience may somehow supersede the findings of CLT. This may be true but I am sceptical. I don’t think that neuroscience has much to say that is educationally useful and I’m doubtful if it ever will.

Finally, Leaton Gray’s argument migrated to the sample sizes used in CLT research:

I disagree with this for three reasons.

Oddly, really large sample sizes are friendly towards dodgy results. The way that calculations of statistical significance work means that if you make your sample size larger you are more likely to find a statistically significant result.

Secondly, small randomised controlled trials (RCTs) have their advantages: They are far less likely to be confounded. So far, the Education Endowment Foundation (EEF) have a strong record of testing the effect of doing nothing against the effect of doing something. Yet doing something usually involves a whole package of things. For example, if we give extra reading tuition then are we measuring the effect of the type of tuition or the effect of simply having more of it? Given the kinds of expectation effects that plague education research, we could find ourselves spending millions studiously measuring the effects of various kinds of placebos.

Finally, the EEF don’t usually randomise at the level of individual students. If you have 3000 students participating in a study across 30 schools and you randomise at the school level then the size of your sample is actually 30 rather than 3000.

Where is the evidence to support differentiation?

Following my last post, a number of people challenged me on Twitter about differentiation. I was not surprised by this. Differentiation is an article of faith within schools of education and questioning it is seen as heresy.

Let me make clear that I am not against all forms of differentiation. Part of the problem is the elasticity of the term. I have had discussions with some who regard the practice of placing students in different classes based upon prior achievement as ‘differentiation’. The evidence for ability grouping is ambiguous but at least it has the potential to be efficient. I also employ practices that could be described as ‘differentiation’ in my own work. For instance, I often ask some students to work on problems while I reexplain a concept to a section of the class.

Yet to many teachers and teacher educators, differentiation means something quite specific. It means grouping students within a class and planning different activities for these groups as well as perhaps providing options for students. For instance, this article in The Conversation describes a process of differentiation that allows students to demonstrate what they have learned through alternative means:

“Actually, differentiation is about teachers providing choice to avoid discriminating against students who may be disadvantaged by ‘one size fits all’ approaches…

It does mean that students with expressive language difficulties or dyspraxia, for example, who experience difficulties with writing are given the opportunity to demonstrate their learning in other ways.

This was famously demonstrated in the movie The Blind Side where Michael Oher’s teachers conducted oral assessments, enabling him to successfully demonstrate his learning.”

I think this is a flawed idea. If students struggle with writing then doing less writing than their peers hardly seems like a plan to address this. Over time, if this kind of differentiation is the norm across classes then we should see a widening gap in writing ability between the child with language difficulties and his or her classmates. Perhaps we intend to just give writing away? I’m not prepared to do that.

I am also of the view that within-class differentiation is impractical to plan and inefficient to deliver. It requires a teacher to manage multiple groups engaged in different tasks. In a sixty minute lesson with a class divided into five groups, the teacher is able to spend a maximum of twelve minutes with each group. And that’s before you take out all the time spent just managing the arrangement; “The table at the back need to focus while I talk to these guys at the front. Thank you. Wait, why are you wandering around, Carl?” These factors are likely to be part of the problem with the L3 programme in New South Wales.

So I have reasoned arguments against differentiation and I think they are valid ones. However, I also have some empirical evidence. A large scale study was carried out across a number of states in the U.S. One of the authors of the study was Carol Ann Tomlinson of the University of Virginia, probably the world’s greatest proponent of differentiation. So you would imagine that they gave differentiation a pretty good go. Yet they couldn’t get it to work. This may be because it was hard to train teachers to do it correctly, it may be because differentiation is flawed in principle or it may, as I suspect, be a bit of both. It’s not promising, however you look at it.

If we accept that whole class teaching is the default and that differentiation involves teachers doing something different, then the burden of proof actually lies with differentiation’s proponents. This burden appears even greater when you consider that every school of education teaches differentiation as best practice and it is required by the Australian Professional Standards for Teachers. You would think there must be tons of evidence. So it should be straightforward – my arguments should be easily falsified by citing studies that show that differentiation leads to enhanced educational outcomes.

The most popular framework for differentiation in Australia seems to be Universal Design for Learning (UDL). I have my doubts about this model but an advocate of differentiation pointed me to a paper with the promising title of, “The effectiveness of universal design for learning: a meta-analysis of literature between 2013 and 2016.” Perhaps this would conclusively prove me wrong? Perhaps it would demonstrate clear evidence for the effectiveness of this form of differentiation? The abstract states:

“Universal Design for Learning (UDL) is often promoted as an inclusive teaching methodology for supporting all students within diverse contemporary classrooms. This is achieved by proactively planning to the edges of a classroom by thinking of all the potential needs of students. To examine its effectiveness, a meta-analysis was conducted on empirical research, containing pre- and post-testing, published in peer-reviewed journals between 2013 and 2016 (N = 18). Results from this analysis suggest that UDL is an effective teaching methodology for improving the learning process for all students. The impact on educational outcomes has not been demonstrated. The implications of this study will be discussed.” [my emphasis]

The paper lists many positive experiences of using UDL and positive effects on how teachers and students feel, but the evidence for impact on educational outcomes is indeed lacking. This is a problem because it is possible to make students and teachers feel positive about pretty much any innovation. What we really need to know is that it leads to more learning.

It was also suggested to me that lots of teachers use differentiation every day and find it effective. How can I dismiss these teachers and their experiences? Do I not value their opinions? Unfortunately, testimonials are an unreliable form of evidence. If I were able to produce a list of teachers who said differentiation does not work for them – which I reckon I probably could – then would this refute such an argument? I suspect I would be told that they must not be doing it properly.

Finally, I was linked to a piece that attempts to explain why teachers are sometimes resistant to differentiation. It’s fascinating. I was particularly taken by this claim:

“Research clearly indicates that students will typically perform better on standardized tests when they have had the opportunity to learn in preferred modes, even if the test is not in their preferred mode. “

Unfortunately, there is no link to this research or a reference to follow. Preferred modes of learning are known more commonly as ‘learning styles’ and this statement expresses the ‘meshing hypothesis’ that matching instruction to a student’s preferred learning style is more effective than not matching the instruction in this way. This hypothesis has been extensively investigated and no supporting evidence has been found.

There are plenty of educational strategies that, like differentiation, lack a solid evidence base. That’s not necessarily a problem. Teachers should be presented with these strategies and a fair appraisal of the research. They may then – individually or as part of a whole-school approach – make informed decisions about which ones to apply.

What I object to is the enforcement of differentiation through teacher standards and the misleading claims that it is best practice or that teachers who don’t apply a particular form of differentiation are discriminating against groups of students. That is coercive. It might be justified if the evidence for the effectiveness of differentiation was strong but it clearly is not.