Understanding the PISA 2015 findings about science teaching

I have shared the following graphic a few times. It shows that frequent use of enquiry-based learning, as defined by the Organisation for Economic Cooperation and Development (OECD), is associated with worse scores on the science component of the Programme for International Student Assessment (PISA). It is based upon surveying students about their experiences in science lessons and then matching this to PISA science performance:

I often point this out to advocates of inquiry learning. Nobody likes cognitive dissonance and so the response I usually receive is, “Well yes, everything done to the extreme is a bad idea. This just tells us about those students who are exposed to inquiry learning in all or most lessons.”

So it is worth going to another chart. This chart shows how an ‘index’ of enquiry-based learning affects overall results. The index isn’t just about enquiry in all or most lessons but about the relative amount. As you can see, more ‘enquiry-based learning’ is associated with worse science PISA results.

You will note that one of the factors that is associated with better PISA science scores is the ‘index of teacher-directed instruction’. Another is the ‘index of adaptive instruction’. This latter term is explained by PISA in the following quote:

“PISA asked students how frequently (“never or almost never”, “some lessons”, “many lessons” or “every lesson or almost every lesson”) the following happens in their science lessons: “The teacher adapts the lesson to my class’s needs and knowledge”; “The teacher provides individual help when a student has difficulties understanding a topic or task”; and “The teacher changes the structure of the lesson on a topic that most students find difficult to understand”. The index of adaptive instruction combines these three questions to measure the extent to which students perceive that their science teachers adapt their instruction based on students’ needs, knowledge and abilities.”

So students seem to do best with explicit teachers who respond to feedback. This is not surprising given the wealth of research on explicit instruction that shows exactly this. And teaching is all aided by a positive discipline climate where students are not hindered by the behaviour of others. Who would have thought it?

If you want to see what the ‘index of enquiry-based learning’ pattern looks like between countries then I have created the following chart from the available data:

That’s quite a strong negative correlation.

It’s worth reminding readers that PISA is a test of application and not of simple recall. Many of the test items require students to evaluate experimental designs and so on. This is why they use the dreadful term ‘scientific literacy’ to describe it. Andreas Scheicher who heads the programme is not an educational traditionalist.

All data and graphics were obtained from Volume 1 and Volume 2 of the reports that can be found here.

Advertisements

7 Comments on “Understanding the PISA 2015 findings about science teaching”

  1. Looking at the first figure it seems to me that it is really showing that practical work doesn’t contribute much to science knowledge. This is something I am aware has been shown by other research into science education. I wonder whether the questions PISA used to investigate “enquiry learning” measure what they intended. Also I am a little perplexed by the second figure and how the countries with the highest index of enquiry based learning are not those I would have thought were bastions of such a progressive teaching approach. Maybe they just do a lot of practical work in science lessons or maybe there is something else going on causing the students to answer strongly on those questions in those countries. Great supporter of your blog, but skeptical that this evidence is a strong argument against enquiry learning.

  2. Michael pye says:

    I noticed the oecd effect seemed to be much larger for most could you explain this. Also there are a lot of of interesting results on the chart.

    Does the index of educational resources mean more is correlated with better performance or the other way around.

    Same with index of educational leadership, private schools and teacher proffession development though they all seem quite weak.

    The big one that got my attention was homework or extra study time. This seems to say it has an extremely negative effect.

  3. […] of the intriguing features of the most recent round of PISA results was that practical work seemed to be negatively […]

  4. Only just delved into this so I’m late to the party here, but I started having a look at the reports that accompanied the PISA results, and from what was written you would think that inquiry based learning and general progressive ideology-informed teaching and learning was associated with higher scores. The three reports on high performing countries Singapore, Estonia and China all seemed to be centred on how their curricula promote the teaching of 21st C skills and a general move to more progressive ed (especially in science). Was pretty astonished to see that about China in particular! What’s going on here?

  5. […] This is not the first instance of strange pronouncements from PISA. The overwhelming finding from PISA 2012 was that their measure of ‘student-orientation’ correlated negatively with PISA results in every country. And the story from PISA 2015 was that the more ‘enquiry-based’ science students were exposed to, the worse their results. […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s