Embed from Getty Images

I remember well the day at the Cavendish Laboratory when the truth sunk in; the enormity of it. As a first year undergraduate, for every single calculation I would make, I would be expected to perform a parallel, often longer, calculation to estimate the uncertainty in the figure I had just produced. Uncertainty is that fundamental to science. Given that all but the most basic treatment of uncertainty is often left out of the school curriculum – probably for motivational reasons – we might even use an understanding of uncertainty as a proxy for a scientific training.

Uncertainty stalks scientists like a black dog on the moors.

The early twentieth century saw uncertainty woven into the very fabric of nature. Rather than uncertainty representing our imperfect knowledge, the new field of quantum physics was intrinsically probabilistic. If we knew everything that there was to know about the universe and we crunched that data on a reasonably large computer, we would still not be able to predict the future.

Heisenberg’s uncertainty principle neatly captures this. If we know exactly where an electron is then we can know nothing about its speed and the direction it is moving. If we know exactly an electron’s speed and the direction it is moving, we can know nothing about where it is. In practice we deal in trade-offs; a fuzzy speed, direction and position. This hasn’t stopped quantum physics and its descendants from predicting experimental results extremely accurately and it hasn’t stopped quantum physics from having important practical applications.

If nature is uncertain at this fundamental level then if remains uncertain all the way up. We may not be able to predict the exact movement of individual ink molecules in a droplet, but if we add this droplet to a tank of water then we know that the molecules are likely to spread out and that we would need to wait many times the age of the universe in order for them to happen to clump back together again. This is an example of the second law of thermodynamics and it demonstrates that, without being able to predict the behaviour of individual members of a group, it is possible to make predictions about the group as a whole.

We also see uncertainty everywhere in medicine. There are genes that can increase your probability of developing cancer. And most medical treatments are probabilistic. A randomised controlled trial (RCT) for an antiviral might reveal that taking the drug reduces, on average, the duration of flu or the chance of hospitalisation, but it will never guarantee any outcome in an individual case. A flu vaccine, on the other hand, may reduce your chance of contracting the virus this season by, say, 40-60%.

Education interventions are also assumed to be probabilistic. For instance, Response To Intervention (RTI) is a method for teaching early reading. It posits the use of high quality programs to initially teach children to read with tiered interventions then kicking-in for students for whom the initial approach did not work. This is consistent with the approach of a medical intervention even if, for now, we don’t have the same number and quality of RCTs to draw upon in education.

And all of the above is why I don’t buy the idea that medicine and education have a different ‘ontology’. Consider the following audio comments by Professor Trevor Gale about his book chapter ‘What’s not to like about RCTs in education?’, a paper I discovered via this AARE blog that was the subject of my previous post (I have transcribed the audio myself so apologies for any errors):

“Because in medical science they have a particular view of the world – what is true, what is reality – they draw on a very physical world… the world of education is dealing with the social world… This world operates and can operate differently to the physical world… If we are looking at RCTs, there’s a claim that there is a cause-effect relationship between things, so… if you do xy will happen, and that’s really the test. So, if I inject you with this serum, you’ll lose your chickenpox or whatever you have or it will protect you from getting some strange virus when you are in Africa. So, there’s like that element of certainty that I think, in the social world, there are some things that look like they are certain, but they’re not always and they can change in different contexts… It would be wonderful if we could just say, ‘you teach this, students will learn that,’ and that there is this direct line that goes between the two, but having been a teacher… they know that that’s not always the case.”

There are many valid arguments about randomised controlled trials, even within the field of medicine. But this is not one of them. It is basically a misconception about science and about the role of uncertainty in science and in fields such as medicine. The fact that RCTs cannot predict exactly what will happen to each individual does not invalidate their usefulness in education any more than it does in medicine.

I’ve noticed this misconception before, chiefly in the writing of Gert Biesta, but I am now concerned that it has grown legs and started to wander about the place. It is therefore important that those of us with a scientific background communicate the fact that uncertainty is fundamental to science and that a wholly deterministic relationship is not required in order for an experiment such as an RCT to provide valid and useful results.


8 thoughts on “Uncertainty

  1. Entirely agree with your view of uncertainty. But in that sense, medicine is more like education than the opposite. There are still differences, however, in that one is a pre-emptive, speculative process and the other is a reactive, remedial one. That makes quite a difference to the significance of uncertainty in the two fields. However the real problem comes not in the uncertainty of research so much as the fact that this gets lost in the way it is presented in education. The most common ‘sense’ is that we are being presented with incontrovertible ‘knowns’ which we are then ‘directed’ to apply, rather than possibilities for reflective consideration. The fault here lies too often with a management culture that is in constant need of certainties and quick fixes as a result of the pressures on it to ‘deliver’. It distorts and damages the whole process to claim that it is certain, knowable and mechanistic in application.

    1. I think you may have missed the point. Good education research can compare well defined interventions and tell us which ones work best for specific outcomes. On it’s own this isn’t good enough but if we can triangulate with other proxy outcomes and with coherent theory then we have a good basis for what makes up the heart of good teaching. The issue is that this is frequently the very different to what our teacher training, our managers and just as importantly our staffroom colleagues think is good teaching. Every colleague in my department (who I get on with) will listen to my arguments but will then agree only with those points that match their experience. The only colleague who every researched these arguments has since retired. We do this to ourselves.

  2. I find that a very strange attack on RCTs by Gale and in the previous post by Brian – maybe I have been thinking of them all wrong. I thought we want RCTs precisely because things aren’t always a cause and effect scenario and we want to find ‘most probable’.
    If you are looking at the effect of a drug on a cell I don’t see why you would need a RCT. But a drug on a person could have a probability instead because there are more factors affecting success.

  3. How many of the people who are concerned about the worrying uncertainty of RCTs are equally worried that their own research is wrong?

    It would be highly inconsistent to argue about ignoring well prepared experimental results “because uncertainty” and then be sure that your own results, using less rigorous methods, are right.

    1. I’m not sure that the favoured forms of research can be wrong. That would mean they are falsifiable and I think these guys reject that whole falsifiable positivist paradigm because they don’t like the look of the ontology. It’s about rejecting binaries or something.

  4. Nice find Greg. Very sad that the piece you quote passes for worthwhile thought to some. It’s interesting that he seems to see the social world as not part of the physical world. I’m all for enjoying fiction but even that has to get to me in some medium in the physical world.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.