The results for the 2018 round of the Programme for International Student Assessment (PISA) will be released on the third of December. What will they show?
I don’t know, I’m not a fortune teller. However, it will be interesting to see if certain trends, both numerical and rhetorical, will continue. Will Scotland, Australia and Finland continue their general downward trends?
Scotland’s PISA results. Note that the average for countries (sort of) is 500 and that no, it does not make sense to start the y-axis at zero
What about England? Presumably, this will be the first cohort of 15-year-olds impacted by the new, harder GCSE exams. Will that have an effect, no effect or is it too early to tell?
I am mainly interested in trends within individual countries because I do not think the rankings tell us very much and can even mislead. For instance, when the U.K. and Australia gained higher rankings in the 2015 assessment of ‘collaborative problem-solving’ than they had in the more standard English, maths and science tests, some saw this as evidence that these countries may not be great at the traditional subjects but were far better at stuff that really matters. However, fewer countries took part in the collaborative problem-solving assessment and the rankings of the U.K. and Australia were pretty similar to their English, maths and science rankings when you took this into account.
And I mentioned the trends in rhetoric. It will be interesting to see how long the OECD (who run PISA) can remain at odds with reality. It has the unfortunate habit of proving itself wrong, with possibly the most amusing example being its advocacy of a ‘student-oriented classroom climate’, when its own survey data suggests this is ineffective*, at least in terms of how the OECD have defined it.
The signs are that Andreas Scheicher, Director of the OECD Directorate for Education and Skills, is still setting up pointless dichotomies between knowing the mechanics of maths and like really deeply understanding it, dude. So perhaps he is going to stay the course.
Yet, how much longer can this go on? If the results show further declines for favoured Finland, East Asia continuing to perform highly and perhaps some surprise – and disappointing from the OECD’s perspective – results that push us into new territory, won’t they have to start changing their tune?
Interestingly, the top officials at the OECD are likely to already know the results. They will be tight-lipped about exactly what these show but it might be interesting to look for any changes in their rhetoric over the next month or so.
*To the extent that the correlational data provided by PISA can suggest anything – these are not randomised controlled trials
Filling the pail 
I am no fortune-teller either, but I’ll make one prediction which I reckon is pretty safe: no matter what the results actually show, Andreas Schleicher will still end up singing from the same hymn sheet.
I am a fortune teller.
And in 2021, we will introduce a new framework to test math in a way that more accurately reflects the role and nature of math in our world
They don’t like the results — so they are going to fix them. In the way bent bookies “fix” things. They think they only have to stay the course until then.
Unfortunately, they’ll find that it won’t work. Because you can’t use “big data” or technology well if your underlying understanding is poor. And underlying understanding requires a good grasp of fundamental techniques.
But we’ll have a new baseline, so it will take a few more rounds before it becomes clear that the countries that use inappropriate techniques will still continue to fall.
Hmmm… that’s a very interesting point.
Reblogged this on From experience to meaning… and commented:
Maybe this is an example of the change Greg is suspecting:
Having in mind that PISA scores only show deviation from the mean of all countries, I can’t see how the historic flow of one particular country could show anything at all. We’d need raw, absolute scores to make inferences with meaning, but those data are esoteric. IMHO, PISA exists just to stimulate country-to-country rivalry(and the subsequent news maelstrom), as educators we can obtain little benefit from there.
So when a country introduces new methods, and its scores start to drop, that shows nothing at all?
How about all the correlational data that PISA also report? That’s not affected by deviation from the mean.