PISA results are in…

I have made a few predictions about PISA both on this blog and on Twitter. I predicted an entrenchment of East Asian countries at the top of the table, a decline for Finland and perhaps a decline for Canada. These were all based upon my interpretation of the policy decisions they have been making and on recent trends.

East Asia is more mixed than I predicted. Singapore has moved further ahead but there have been some declines in Vietnam and Korea. Finland has fallen away again, as predicted, and Canadian maths scores have dropped by another 4 points – these both add to longer trends.

I also made predictions about collaborative problem-solving but it appears that the results for these tests won’t be out until 2017.

I didn’t realise that survey data was going to be published alongside the science results and this is fascinating. PISA designed a set of questions to determine how ‘teacher-directed’ or ‘enquiry-based’ the science teaching is.

The data shows that the greater the level of enquiry-based science instruction, the worse students’ science scores and that the greater the level of teacher-directed instruction, the greater the science scores. In PISA’s words:

“Perhaps surprisingly, in no education system do students who reported that they are frequently exposed to enquiry based instruction (when they are encouraged to experiment and engage in hands-on activities) score higher in science. After accounting for students’ and schools’ socio-economic profile, in 56 countries and economies, greater exposure to enquiry-based instruction is associated with lower scores in science.”

I don’t find this surprising at all. There are important reasons why inquiry learning does not work very well.

Yet, just last weekend, an increase in the amount of inquiry learning in science classes was being suggested in The Conversation as the solution to stagnating TIMSS scores.

We need to move away from strategies that we prefer on philosophical grounds in favour of ones that are the most effective.

Links:

http://www.oecd.org/publications/pisa-2015-results-volume-i-9789264266490-en.htm

http://www.oecd.org/pisa/pisa-2015-results-volume-ii-9789264267510-en.htm

 

Advertisements

24 Comments on “PISA results are in…”

    • The pedagogy of the Academics!

      An excerpt from a conversation today from an education academic to student teachers:

      Based on strong research on how children learn, constructivist pedagogy is a better than direct instruction. And we recommend that you use it in your classrooms.

  1. […] Source: PISA results are in… […]

  2. One paragraph later on the same page (p. 38 of PDF version of Policies and Practices for Successful Schools), they seem to contradict the earlier statement with this:

    “Students perform better in science than in [reading and mathematics] when they spend more time learning science than learning the other two subjects […], and particularly when their teachers frequently explain and demonstrate scientific ideas, support students in their learning and expose them to more enquiry-based instruction.”

    So if they are “frequently exposed to enquiry based instruction” they don’t seem to do better, but if they are “exposed to more enquiry-based instruction” (together with the other factors) they perform better in science than in reading and mathematics.

    Do these two findings pull in opposite directions or am I missing something?

    • Greg Ashman says:

      The graphs in the report (vol II) show what they asked and how this correlated to performance.

    • lemieux2016 says:

      I speak from the perspective of a retired Science teacher (grades7-8). Without looking at the graphs in the report to which Greg refers, I am thinking the following: when I was given four 40 minute periods per 5 day cycle in which to teach Science, and had freedom to make at least two of those periods I to a double period, I was able to teach using more teacher guided inquiry approaches and using hands on investigations, because I had enough time to take up the investigations and base my much-needed direct instruction on the data the students had collected during the investigations. I could not only teach the Science content, but teach how Scientists use evidence-based reasoning. In the years when my teaching time was cut back to 3 single 40 minute periods, I had to forego a lot of hands on, investigative learning in order to have time to do enough direct instruction to ensure students understood the content. This meant they had less opportunity to develop the scientific skills and processes that were included in the curriculum. If I had continued to give the same time to hands on investigations as before, then the students would not have been able to learn from their findings. They needed the after stage when I helped them to make sense of patterns in the data, for example, and understand how their results supported the theories, and illustrated the concepts and principles that were at the heart of the content of the curriculum. In a PISA or any other test, students are not, I take it, required to plan and conduct investigations, analyze and make conclusions based on their results, and discuss possible sources of error, etc. So, if I wants only for them to do well on such tests, I would make sure to spwnd every moment directly teaching the content.
      Perhaps what the report indicates then, is that when teachers have less time to teach science compared to English and math, if they spend comparable more time in inquiry teaching ( frequently exposed to inquiry learning) compared to students in other places, the students have difficulty learning the science content. Alternatively, if students spend more time in science than in English and math, and teachers use some of that time to teach through inquiry and are able to support students in understanding the content through their investigations, the science content learning is effective.
      The devil is always in the details. I mentor young science teachers. One I currently support teaches 7 Intermediate grade classes ( 30 plus students per class) in a 5 day cycle and has 3 single 50 minute periods for each class in that time frame. I advised her to severely limit how much hands on learning she does in order to have time to adequately directly teach and explain the concepts etc . A huge factor in helping students understand science is having time to assess their learning as they go, and intervene in a timely manner to correct misunderstandings. When one teaches 210 students, managing hands on learning and following it effectively with direct instruction and assessment that monitors learning is a mind boggling task. It is much easier to directly teach and use easy to correct assessment tasks, and wil more likely result in students performing better on tests of content learning.

  3. monkrob says:

    We also need to decide if the main purpose of our education system is to get students to do well on PISA pen and paper tests.
    Looking forward to the problem solving tests. if the correlation is the same there, R.I.P. enquiry learning.

  4. Richard I says:

    I especially like the multiple regression analysis that suggests skipping classes has less of a negative effect than enquiry-based instruction.

    http://www.keepeek.com/Digital-Asset-Management/oecd/education/pisa-2015-results-volume-ii_9789264267510-en#page230

  5. […] Greg Ashman has explained here, this probably shouldn’t be all that surprising. That said, enquiry does seem to positively […]

  6. lemieux2016 says:

    I looked at page 73 and the chart Greg referred to, and was very surprised at the inclusion under inquiry teaching as whether the teacher ” explained how a science idea can be applied to a number of different phenomena.” Isn’t that what any good teacher would do who wants to help students learn Science content effectively ? I am guessing that a good teacher who claimed to only teach through direct instruction, would directly instruct students in understanding important connections to other ideas and applications as part of their content instruction. The results show that that that component of inquiry learning positively correlated to achievement, whereas opportunities to do hands on lab work and design own experiments did not, and I shared my thoughts about the possible causes for that latter result in a previous post.

  7. David says:

    I like this: “Across OECD countries, the more computers available for educational purposes per student, the lower students score in science, but only before accounting for the socio-economic profile of students and schools. There is a similar number of PISA participting countries and economies where the relationship is positive (7) as education systems where it is negative (11), after accounting for the socio-economic profile of students and schools.”

  8. Joe says:

    I’m sorry, but does Australia only teach science as inquiry?(I’m American)

    Is there a country out there where every science teacher has perfected the craft of inquiry and solely uses the inquiry technique?

    Also, you really take this test seriously? Again, a major problem with these BS standardized tests is that it is assumed that everyone that takes them tries their hardest. Yeah, kids always try their hardest on tests that have no effect on their grade or life.

    I’ll leave you with this from http://curmudgucation.blogspot.com/2016/12/write-your-own-pisa-post.html

    The PISA is an international test that serves to measure educational achievement in nations that don’t even speak the same languages. Its validity and accuracy has been established by
    a) the organization that created it
    b) tiny invisible accountants
    c) hopes and dreams
    d) insisting real hard repeatedly

  9. viktorblasjo says:

    Regarding enquiry-based learning having a negative effect, one may note in the same table that “after-school study time” is strongly associated with negative scores. Surely this is a matter of correlation rather than causation: students who spend a lot of time studying outside of class are weaker on average, but this is not the reason they are weaker, one would hope. Could something like this be the case for enquiry-based learning too? This would have to mean that weaker students are exposed to enquiry-based learning to a greater extent. Is there any evidence for this? I’m not sure.

    More confusingly, it seems the dichotomy of direct instruction versus enquiry-based learning is a false one, according to this data. Because many countries do either lots of both or very little of each (Volume II, pages 64 and 72), which makes no sense if we are picturing it as and either-or situation. Korea, for instance, are dead last by a wide margin in the use of teacher-directed instruction, yet they are somehow also second to last on enquiry-based learning. What on earth are these Koreans doing in their classes then, if it’s neither one nor the other? Many other countries exhibit similarly paradoxical results. This suggests that this data is poorly suited for making judgements about one teaching style versus the other. For this purpose, it would have been better to have asked the students questions that forced them to pick a point on this continuum instead of asking them about each separately, in a non-exclusive way.

  10. dodiscimus says:

    Two points before I start: I’m not overly keen on enquiry-based learning and have spent most of my own time in the classroom using practical work mainly to support direct instruction and keeping discussion of socio-scientific issues pretty teacher-led; I also think the weight of evidence already generally supports direct instruction, at least over a heavily enquiry-based curriculum in school science.

    However, I think some care is needed in interpreting these PISA results. I may have missed it but cannot see any reference to testing of the validity of the student report items. There are some items that look ambiguous to me, but regardless of the face validity, it is an issue if no-one has matched student responses on these items to independent observation of their science lesson experiences. I also wonder whether the between-country differences are actually greater than the in-country differences and I don’t know what to make of e.g. the USA and Russia reporting high levels of both teacher-directed and enquiry-based instruction, whilst e.g. Korea reports low levels of both. It certainly doesn’t look as though the two approaches are dicotomous, even if they are not actually symbiotic.

    A further point, brought up in the report, is that if the correlation is causal it could be the other way around. Higher-achieving children may encourage their teachers to pile on the direct instruction whilst lower-achieving children may get more practical and ‘engaging’ enquiry-based activities. If you already believe that this is the soft bigotry of low expectations at work then it doesn’t really affect the interpretation but if you think this may be a reaction to the expectations of children (and parents) and also bear in mind that lower-achieving children tend to get less effective teachers etc etc it starts to look like a more complex picture.

    Finally, it seems as though both higher levels of teacher-directed and higher levels of enquiry-based instruction lead to a better understanding of the nature of science and to greater motivation to follow STEM careers; it’s just PISA test scores that only follow the teacher-directed index.

    To my mind it is another small piece of evidence in favour of direct instruction, but just because it comes from a very big and expensive piece of research doesn’t make it definitive, any more than the success of DI for KS1 children in the large and expensive Project Follow-Through tells us anything definitive about teaching teenagers.

    Best wishes

    • Greg Ashman says:

      No single source of evidence in education is perfect and I accept the limitations of the PISA study. However, when you triangulate it with all of the other evidence we have from teacher effectiveness research, experimental trials and cognitive science, and when these all point in the same direction, I think this is highly suggestive.

      Let’s ask the opposite question – where is the large body of evidence supporting inquiry approaches?

  11. dodiscimus says:

    There is no large body of evidence supporting inquiry approaches for novice learners.

    That is why, by and large, I encourage my trainee teachers to explain clearly, question deeply, and get the children to practise lots, before applying in increasingly varied contexts. But I still think there is all sorts of fuzziness round the edges with a lot of individual decisions to be made by individual teachers about individual classes. Teachers should be very aware of the evidence but I don’t buy arguments that there is one right way to teach for everyone all the time, and I think that’s where my caution stems from.

    • Greg Ashman says:

      I don’t think anyone is making the case you argue against. Even I accept the value of conducting practical work. I am very concerned that teachers don’t seem to know that – in general – explicit instruction is highly effective and that inquiry lacks evidence.

      • dodiscimus says:

        I think you are right. What is hanging around in schools still is the remnants of the very constructivist elements that were promoted by various sources (including ITE) and rather driven home by all those Ofsted expectations of student-led practice. I find a lot of school mentors are caught between what they actually do and find effective (which in science is usually a mix but with a traditional slant), and what they remember and think they ought to be encouraging trainee teachers to do, hence advice to trainee teachers about reducing teacher talk framed as best practice rather than because an explanation wasn’t succinct, or instructions were too complicated. It’s a potentially endless cycle because most people involved in ITE (including me) are drawing heavily on practical experience, which includes this tendency to stick with old ideas. The cycle is gradually being broken, though, and there will be a tipping point. What I don’t think we (I) should do is get trainee teachers tanked up and evangelical about the evidence for explicit instruction in the couple of weeks we have before they are mainly in school, and then send them crashing in to work with highly effective teachers who will have a range of approaches. I don’t want to instigate a revolution; half-revolutions are preferable.
        Best wishes

  12. […] decline in recent years in international comparisons. In its report on the 2015 assessments, PISA admits that enquiry-based, i.e. progressive, education does not work. So far Ireland has held out against […]

  13. […] Source: PISA results are in… […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s