A friend drew my attention to a blog post which led to me tracking down the study that the blog post is written about. I think this study is of importance to those of us with an interest in education research.
Trials of new drugs tend to be double-blinded. Patients are either given the test drug or a placebo – a simple sugar-pill which looks the same. The patients do not know which of these they have been given and neither do their doctors (hence the ‘double’ blinding). Part of the reason for such secrecy is to reduce expectation effects – patients will all have an equal expectation of getting better.
This is why it can be hard to evaluate complementary therapies such as acupuncture – you tend to know if someone has stuck a needle in you. If we compare acupuncture with, say, massage then people might believe that the acupuncture will help them more than the massage and this may affect the results.
In education, we often refer to an expectation effect known as the ‘Hawthorne Effect’ where the knowledge of being the subject of study somehow affects the results – perhaps teachers are more enthusiastic or perhaps students work a little harder. Again, this is an expectation effect and it is one of the reasons why we should be skeptical about research. If teachers opt in to teaching the new, shiny program and students are aware that they are in this program – or even self-select into it as has been the case with some university-level trials of problem-based learning – then should we really be surprised that there is an effect when compared to business-as-usual?
In fact, a new, shiny program would not only involve a Hawthorne effect – the knowledge that you are the subject of study – but a placebo effect – a new method or piece of equipment or construct of some kind that may cause you to expect a favourable outcome.
Should we worry about these effects? After all, expecting to do better won’t teach us how to read and it seems unlikely that it could have an effect on something as fundamental as our underlying intelligence.
This is where the new experiment comes in. The researchers were investigating the possibility of a placebo effect in ‘brain-training’ activities. They recruited participants to the study using two different posters. The first was neutral and asked student to “participate in a study” in order to gain course credits whereas the other poster – the placebo condition – mentioned “cognitive enhancement” and the potential to “increase fluid intelligence”.
The intervention was then the same – a brain-training game. So it wasn’t the intervention that was manipulated but the recruits’ expectations of it. It’s a bit like giving acupuncture to two groups of patients but promoting the potential benefits with only one of the groups.
There is a possibility of a systematic difference between the two groups given that assignment was by self-selection rather than being random. Even so, performance on the brain-training task was similar and yet students who self-selected into the placebo group saw the equivalent of a 5-10 point increase on a standard IQ test.
This feeds the ongoing debate about whether brain-training achieves much at all and, if it does, whether this transfers out of the narrow range of skills that the training addresses. But it should also give us pause to reflect on educational research. Has anyone ever run a trial of an educational placebo?