PISA 2030

Embed from Getty Images

Let’s remind ourselves of how the Organisation for Economic Cooperation and Development (OECD) define effective teaching in their Programme for International Student Assessment (PISA):

“In its Analytical Framework (OECD, 2013), PISA defines the three dimensions of good teaching as: clear, well-structured classroom management; supportive, student-oriented classroom climate; and cognitive activation with challenging content”

This is a problem for PISA and its head, Andreas Schleicher. Why? Because data from PISA consistently proves that a ‘student-oriented classroom climate’ as defined by the OECD, does not represent effective teaching. In both 2012 and 2015, practices associated with this definition were linked to worse outcomes on PISA assessments.

Schleicher and the OECD have therefore been aware of this paradox for some time. Rather than rethinking their commitment to a ‘student-oriented classroom climate’ or the way they define this, they appear to have been casting about for different measures that support their original view.

For instance, in 2015, PISA introduced an eccentric test of ‘collaborative problem solving’. Perhaps all those student-oriented activities may have been detrimental to academic outcomes but would prepare students better for this assessment? No. The same countries that top the convention PISA tables also performed the best on collaborative problem solving:

“A comparison of the mean scores in collaborative problem solving, science, reading and mathematics shows that the same countries/economies – Canada, Hong Kong (China), Japan, Korea and Singapore – are found at the top of each set of rankings. Indeed, scores in the four domains are highly correlated. On average across OECD countries, student performance in collaborative problem solving shows a correlation of 0.77 with performance in science, 0.74 with performance in reading, and 0.70 with performance in mathematics.”

How tiresome.

Never mind, perhaps a complete overhaul of the way PISA is run may do the job.

This seems to be the thinking at the OECD, with the Sydney Morning Herald reporting major changes ahead which involve, “…moving away from traditional knowledge testing and adding more problem solving elements to the literacy, numeracy and science tests, and introducing a new social skills test.” Given that PISA items already attempt to assess problem-solving rather than knowledge recall, I will be interested to see what these new assessments look like.

Demonstrating the fact that he still does not accept the relatively simple point that knowledge is what you think with, Schleicher also commented, “We’re thinking about 2030, knowledge about maths and science is easy to digitise but the future is different, the modern world doesn’t reward us for what we know, we can get that from Google, but how we apply knowledge.”

At this point, I wonder whether the wonks at the OECD are already in possession of some preliminary findings from PISA 2018 and, if so, whether these findings are in any way driving this new push. Unfortunately for Schleicher, attempts to decouple knowledge and skills with the aim of focusing on the latter have repeatedly failed and look likely to fail again. A good problem solver can solve a problem because he or she knows stuff relevant to solving that problem.


2 thoughts on “PISA 2030

  1. Andreas claims that ‘the modern world doesn’t reward us for what we know, we can get that from Google…..’
    Perhaps Schleicher has been adopting some of his rhetoric from Geoff Hinton, the computer scientist and head of Google Brain. Geoff Hinton is well known for his work with Artificial Neural Nets but has run into difficulty when he dismissed the need for explainable AI. Hinton attempted to defend his position by stating “ I’m an expert on trying to get the technology to work, not an expert on social policy. One place where I do have technical expertise that’s relevant is whether regulators should insist that you can explain how your AI system works. I think that would be a complete disaster.”
    In much the same way that Schleicher & the OECD Pisa project has been allowed to dominate education global rankings and policy direction for almost two decades without accountability, Hinton displays a similar attitude.
    The question is; who is liable for societal and individual harms resulting from black box systems whether they arise from so-called education measurement systems or algorithmic flaws. Who does society hold to account: the data scientist like Schleicher, the software / machine, or the company?
    Permitting self proclaimed experts the luxury of divorcing themselves from social or policy implications ignores the fact that technology is not value neutral and that such decisions taken have immense socio-political implications.
    Which government elected Andreas Schleicher to run their education policy?
    It’s time the OECD Pisa project was switched off before further serious harm results.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.