7 tools for thinking about education

I would like to offer some tools that teachers, parents, journalists and others involved in education might find useful. In particular, these tools are intended to help you evaluate the kinds of claims made by presenters at conferences or in newspapers and on blogs.

1. What evidence would prove this claim wrong? 

Imagine that a speaker takes to the stage and claims that a proprietary thinking routine – let’s call it ‘four hops and a ladder’ – leads to ‘deeper’ learning. The presenter shows evidence that includes lots of bar charts and demonstrates that teachers who were trained in this technique used it more often and that teachers and students alike felt motivated by it. You raise your hand to ask a question. You are aware of a study that showed no academic gains for students who used this routine versus those who did not. Ah, the presenter explain, that’s because the test that was used to assess academic gains did not assess ‘deeper’ learning. You reply, pointing out that the test included some complex transfer problems. Again, the presenter is sceptical that these really represent deeper learning. And anyway, the teachers might not have been implementing the routine properly, he notes.

Not only has the burden of proof been reversed (see below), it is hard to think of any way that we could prove this assertion wrong. The absence of evidence for something is a form of refutation but it can never be 100% conclusive. And that’s because the inductive arguments used by science can never give absolute certainty about anything. Advocates will exploit that.

It is therefore worth considering what kind of evidence really would be sufficient to disprove the proposition to the your satisfaction and to the presenter’s satisfaction. If there is a wide gulf between the two then that tells you something.

2. What are students intended to learn from this method?

Often we read about educational activities described in the most breathless terms. We may learn of maths questions so inspiring that they make it on to T-shirts or of “Aha” moments when the lights go on for particular individuals. But can you identify what students are meant to learn as a result because, often, the intended learning is not mentioned at all. All you get is the description of an activity. By definition, education has to involve learning something. So if this is not articulated then there are two possibilities. You have either been presented with an activity that has no educational benefit or the presenter has chosen not to mention what it is. Why?

3. Are you being sold motivation?

One reason that the intended learning might not be mentioned is that you’re not being sold learning at all, you’re being sold motivation. That’s fine as far as it goes but there are many fun activities in this world and many ways to pique student interest. It is all educationally useless if it doesn’t lead to more learning. If this new, motivating approach to teaching grammar leads to more and better learning of grammar then there should be evidence of that and not just evidence that it’s fun.

I am sceptical about generating what the literature terms ‘situational interest’, that is interest in the current activity and moment. I’m sure that it can aid learning but the real goal is individual interest; a long term enthusiasm for the subject. This seems to be at least partly the result of a growing sense of competence. And a sense of competence clearly relates to effective teaching practices that lead to learning.

4. Does the suggested approach sit close to the targeted skill or knowledge?

This is a tool I have thought about more recently. Imagine you want to improve a child’s reading; do you teach him or her a breathing technique or do you use a phonics based intervention. The phonics intervention directly relates to reading and the path of influence is clear; better knowledge of grapheme-phoneme correspondences will perhaps lead to improved decoding.

The chain of influence for the breathing activity is longer and more speculative. Perhaps the breathing activity will reduce anxiety. Perhaps this will then allow the child to better access his or her knowledge. Perhaps this will aid the process of reading. 

If we were going to lay bets then the intervention with the shortest chain of influences would be a good choice.

5. Is the evidence a testimonial?

Education is complex, taking place in varied contexts and with many interacting components. Some people use education’s complexity to argue that the standard of evidence used by science is inappropriate and we should instead draw inferences from the kinds of sources that science largely rejects such as personal experience or anecdote.

Precisely the opposite is true. If someone is presenting you with a method to apply in your classroom then she needs to demonstrate the general effectiveness of this method. The fact that it was perceived to work in a particular context does not provide this evidence.

Scientific approaches such as experimental trials or epidemiological studies have the capacity to provide the evidence for such a general effect. The strongest approaches, such as the use of explicit instruction, can draw positive evidence from a diverse range of trials and studies yet other popular practice, such as certain forms of differentiation, have been around for a long time without generating such evidence. 

6. Where lies the burden of proof?

Arguments are not always symmetrical. If someone is advocating a revolution then they bear a greater burden of proof than those who advocate for the status quo. Current practice might not be perfect but, before we jump, we should make sure we are jumping to something better.

A surprisingly large number of advocates for change simply point to flaws in the status quo. For instance, imagine someone claiming that children leave school with poor problem solving skills so we must give them more opportunity to engage in open-ended project-work. This is a weak argument. 

To strengthen it, we would need some evidence to show that engaging in open-ended project-work will lead to students developing superior problem solving skills. And before we can do that, we need an understanding of what these skills are and how we can assess them. Few gurus are prepared to do this kind of ground work. It’s far easier to decry the present because you can find fault with pretty much anything.

7. Is this an argument from authority?

The Early Years Framework for Australia requires teachers to take account of children’s learning styles. However, this does not mean that the value of taking account of learning styles has been proven. Just because something is a statute or has been asserted by a figure in authority, it does not mean that it is true. In a free society, we may question such ideas.

If an argument rests solely on the authority of the person constructing it, or on an external authority, then this is not particularly persuasive. And such arguments come in many forms. Academics have an unfortunate habit of saying things like, “When you have read as much about this subject as me then you will understand.” Again, this is an argument from authority.

Challenging such an argument is tricky because it may be taken as an attack against the authority in question. So you might want to simply note the argument, factor it in to your thinking and move on.

Advertisement