PISA shows that you can’t always get what you want

You have to feel sorry for the folks at Programme for International Student Assessment (PISA). Like pretty much everyone else involved in education, they have swallowed educational ‘theory’ wholesale and think that they can use this to define the best teaching practices. The problem is that they then set out to empirically test this. The wisest educational theorists never expose their ideas to empirical tests.

Let’s recall the way that PISA defines effective teaching:

““In its Analytical Framework (OECD, 2013), PISA defines the three dimensions of good teaching as: clear, well-structured classroom management; supportive, student-oriented classroom climate; and cognitive activation with challenging content (Klieme et al, 2009; Baumert et al, 2010; Lipowsky et al, 2009; Kunter et al 2008).” [my emphasis]

So , in 2012, they decided to examine the relationship between a student-oriented approach to teaching mathematics and PISA mathematics scores. They created a series of survey questions in order to try to measure the level of student orientation. They found that this correlated negatively with scores – the more student-oriented the teaching, the worse the PISA maths result. The graph below gives some idea and a more sophisticated statistical analysis may be found in this paper.

It is clear what they wanted to find – that a more student-oriented approach led to better results. Unfortunately they found the reverse. However, this finding has not been publicised much by PISA. For instance, in their recent guidance for maths teachers it isn’t really mentioned. Instead, the authors tend to focus on the limitations of teacher-directed instruction which, according to their own data, are far less profound than those of student-orientated instruction.

In 2015, the PISA focus shifted to science. This time, they decided to focus on the role of enquiry-based learning. They created a survey to try to capture this and I imagine that, again, they expected to see it relate positively to science performance.

Unfortunately, it did not. PISA’s enquiry-based science construct actually correlates negatively to PISA science scores across nearly all countries that took part. The graph below shows a good summary:

I wonder what PISA will do to publicise this result? It might not have been what they were after but it does seem to triangulate well with a wealth of other evidence that suggests that teacher-directed methods are better than inquiry-based ones. Perhaps this is the finding that will finally move us past the old trope that we need more inquiry-based science teaching in order to better engage students in science and help them apply their understanding. This trope has been rehearsed recently in The Conversation and, incredibly, in The Age after the publication of the above data.

In the words of The Rolling Stones, “You can’t always get what you want. But if you try sometimes, you just might find you get what you need”.