There were a number of responses on Twitter when Pasi Sahlberg posted a Tweet apparently quoting Diane Ravitch:
At a basic level, the statement contradicts itself. If a student taking the same test at different times will get different results then that student’s results cannot be ordained by their family income and parents’ education. Nevertheless, being charitable, there is a sense in which Pasi and Ravitch are right and we will return to that later.
Looking at the broader picture, there is irony in Sahlberg coming out against standardised testing, if that is what he is doing. Sahlberg is currently professor of education policy at the Gonski Institute for Education at the University of New South Wales in Sydney, but he is originally from Finland. He rose to prominence as an authority on Finnish education, writing books on what the world can learn from Finland. And the world is keen to learn these lessons. Why? In the early rounds of the Programme for International Student Assessment (PISA), Finland gained some of the highest results and this sparked a rush to find out how they did it. But PISA, of course, is a set of standardised tests.
Since those early days, Finland’s PISA performance has significantly declined and the factors often cited as the cause of its early success are likely to be wide of the mark. It still performs relatively well compared to other countries, but this has never been a valid comparison. Countries differ on a variety of factors from the homogeneity of the population, all the way down to the home language and how regular and easy it is to learn. This means that the direction of travel of a particular state or country tells us more than any comparison between different countries.
So in what sense are Sahlberg and Ravitch right about standardised testing? Well, it certainly correlates strongly to family background. In Australia, schools are given an ICSEA score that measures educational advantage. School students also sit standardised NAPLAN assessments in English and mathematics. The correlation between the two is striking (thanks to Julian Rossi, @julianvrossi):
If Sahlberg and Ravitch made the claim that standardised tests are unreliable at the student level but the aggregate scores correlate with family background at the school level, then the claim is more justified. No assessment is ever completely reliable at the student level and therefore accepting their point about reliability depends on how much variation you are prepared to tolerate.
Why would standardised test scores correlate to family background? If all else is equal, it makes sense that children from financially stable homes whose parents are highly educated would do better than those who lack this background, even if there is a fierce argument about how much of this advantage is nature versus nurture. However, in Rossi’s graph, we can clearly see that all thing are not equal. There’s a school with an ICSEA score just over 800 that is far outperforming many schools with an ICSEA score above the average of 1000. I wonder which school this is and I wonder what they are doing?
That’s the advantage of having standardised test scores to consult. We can now ask these questions. Even better, NAPLAN assessments take place at Years 3, 5, 7 and 9 and so, if we wish, we can examine which schools have students who make the most progress.
For instance, Blaise Joseph of the Centre for Independent Studies used NAPLAN data to identify primary schools that are bucking the trend, visited them and described some of the common themes such as strong behaviour policies, explicit teaching and evidence-informed reading instruction. Ideally, it would be good to compare this with a control group of less effective schools*, but nonetheless, this kind of analysis is useful to schools and policymakers who want to know how to improve.
In Part 2, I will examine another context where some schools are bucking the trend.
*You can see why this might be difficult to do. It’s relatively straightforward to call a high-performing school and ask if you can visit them and find out about why they are successful. It is harder to call a low-performing school and ask if you can visit them and find out why they are unsuccessful. Nevertheless, it is feasible that some schools would want to cooperate in order to find ways to improve and this kind of research should be a focus of university education faculties.