The Cambridge-Somerville Youth Study was a ground-breaking piece of research. It followed a mentoring initiative for boys from deprived suburbs of Boston that ran from 1939 to 1945. At the time, this initiative was almost unique in the field of social sciences because it also followed a matched control group of boys who did not receive the mentoring intervention. Through the efforts of researchers, the subjects were traced over a long period of time so that outcomes such as involvement in crime and stability of relationships could be compared.
The study was the subject of a recent Freakonomics podcast that serves as a warning to any of us who wish to intervene positively in the lives of young people. The boys who were given the intervention were overwhelmingly positive about it. And it’s not hard to see why. According to the criminologist, Brandon Welsh:
“The counselors would meet every couple of weeks with the boys, interact with them, help them with homework, take them to the YMCA. During the summer months, some of the treatment-group boys were able to go to summer camps and so were sent out of the city.”
Yet when the data was analysed thirty years later, the boys who received the intervention had fared significantly worse than the control group on a whole range of outcomes related to criminal behaviour, health and work.
Clearly, good intentions are not enough when it comes to social interventions. We cannot fool ourselves into thinking that the worst that can happen is that we have no effect: Sadly, it is possible to do harm. As Denise Gottfredson, professor of criminology and criminal justice at the University of Maryland, states in the podcast, “People just assume that if you do something that sounds good, that it’s going to have positive effects. But it’s actually more complicated than that.” She then goes on to explain that the somewhat counter intuitive result may be due to students in the intervention providing a form of validation for each others’ behaviour.
This is just one example of why there is a clear, ethical imperative to collect quantitative data. We cannot simply rely on our own good intentions or on interventions that are based in theory. It is quite wrong to simply dismiss calls for empirical evidence as, ‘positivism,’ while continuing to intervene with young people in ways that might actually be harmful.
This is why I am so concerned about particular models of differentiation that have become popular in schools. As I have previously written, Universal Design for Learning (UDL) is such a model that lacks evidence to support a positive effect on academic outcomes. One UDL website promotes the program with spurious images of brains. A partner website has a section on the evidence for UDL. However, this section asks visitors if they have any evidence to support these claims. Seeking confirmatory evidence is basically the opposite of science.
One of the features of these forms of differentiation is that they often allow students to present what they have learnt in a variety of different ways. This may be appropriate if a student with a disability simply cannot complete a certain type of task. However, I am deeply concerned about the practice of offering such alternatives to students who have difficulties with a particular task. If a student habitually avoids writing, for instance, then she is never likely to improve at writing and will be severely disadvantaged as a result as the gap grows ever wider between her and her peers. Instead of planning for alternative ways of demonstrating learning, we should be focusing our efforts on intensive and explicit writing instruction.
I might be quite wrong about this, but how would we know?
I have criticised forms of differentiation in the past and it has resulted in some pretty unpleasant commentary. I have been accused by Dr Linda Graham of disregarding the Australian Disabilities Discrimination Act, something I strongly dispute. This kind of authoritarian stance can only serve to silence criticism. Yet as an education community, we desperately need to think more critically. With UDL gaining in popularity to the extent of featuring in an Australian Senate report as an example of good practice, we need to ask whether it is possible, just possible, that it does more harm than good.
I was struck by a post by John Kenny who wrote about a presentation he had recently attended. Advised by the presenter that he should let students with Language Processing Disorder present their learning in the form of a video rather than in writing, he questioned this. According to Kenny, “When I raised this concern with the presenter, she simply stated that it was not fair to make students constantly do what they are not good at, that they should be given the chance to shine at things they are good at.”
It is clear that such views come from a place of deep compassion and concern for students; of this there is no dispute. But what if compassion and concern not enough? After all, they were not enough in the Cambridge-Somerville study. What if, in our rush to apply ideas that we think are sound and that suit our ideological outlook, we are actually doing harm? We will never know if we focus on silencing criticism and refuse to support the kinds of quantitative trials that could answer these questions.