Project-based learning fails a key test

Embed from Getty Images

Back in September, I wrote a post where I asked whether New South Wales should be spending money training teachers in project-based learning (PBL). I questioned its effectiveness and I suggested that disadvantaged children are particularly likely to suffer from a switch to this method.

My post prompted an extraordinary personal attack from one of the consultants delivering the New South Wales training. There was much sound and fury but little light shed on the key question of the effectiveness of project-based learning.

I didn’t realise at the time that the Education Endowment Foundation (EEF) were coming to the end of a trial of project-based learning in the U.K. The report has now been released.

As with all evidence, we have to be cautious. The EEF don’t always report statistical significance. The controversial “Philosophy for Children” trial also appears to have switched its outcome measure from the one originally proposed. I’ve been looking into these kinds of trials a little recently (eg here and here) and there are a lot of potential issues.

The type of PBL that was studied by the EEF is known as “Learning through REAL projects,” and it, “particularly aims to improve their [students’] engagement in learning as well as practical literacy skills.” I cannot see how PBL could improve literacy but it is plausible that it might improve engagement.

Unfortunately, there was no clear impact on levels of engagement. Moreover, the literacy results are very worrying. As a whole, the intervention had no effect on literacy. The effect size was negative but the confidence intervals overlap zero meaning that this could have arisen by chance.

However, the effect on the literacy of children eligible for Free School Meals (FSM) – a measure of disadvantage – is negative and significant. In other words, switching to PBL from a traditional course is harmful for these students.

I haven’t yet read all of the report in detail. A good sign – in terms of validity – is that stratification by FSM seems to have been in the original protocol so this isn’t a post hoc analysis. 

This result is therefore really worrying and adds to a picture of generally weak evidence for PBL. Yet PBL is all the rage across Australia right now.

It is therefore a matter of some importance that people are aware of the research findings. Spread the word.

Advertisements

7 Comments on “Project-based learning fails a key test”

  1. Greg – I wrote this in a blog with respect to engagement (https://3starlearningexperiences.wordpress.com/2016/05/17/close-the-stable-doors-effects-of-motivation-anengagement-on-learner-achievement/):
    Let’s kick it off with engagement. Andrew McConney and his colleagues studied more than 41,000 fifteen-year olds in Australia, Canada, and New Zealand. First, they determined if the curriculum in physics as offered to these learners had a low or high level of discovery learning. Then, they compared the learners’ (1) scientific literacy . They also looked at (2) interest in physics and (3) engagement in physics.
    The researchers were investigating the, by many seen as ‘self-evident truth’ that engagement would lead to better learning; here in terms of scientific literacy. They, however, found that learners who were offered the discovery learning approach showed an above average level of engagement and interest, but a below average level of scientific literacy. In other words, the learners perceived the discovery learning approach as interesting and they were very engaged in their learning. However, they learned less than average. In contrast, learners who followed the ‘boring’ learning approach, showed less than average interest and engagement, but a higher than average scientific literacy!
    McConney and colleagues conclude: “These results seems to be an infringement with the prevailing orthodoxy in the educational world that the more discovery-based learning is, the higher chances are that learners acquire a strong scientific literacy.”
    Repeat after us: This (large!) study has concluded that engagement does not influence learning achievements positively. That’s a bit of a shake-up and we’re not there yet.

  2. This is a really interesting and, unfortunately, rather depressing review. I have long held doubts about the impact and efficacy of PBL and other forms of curriculum delivery in schools. I think it’s important to add the PBL and/or other forms of thematic teaching can be effective, but only if teachers ensure there is a clear focus on the delivery of skills and the development of conceptual knowledge. Unfortunately, PBL and other thematic provision is often characterised by more unguided discovery-based learning and, in these cases, outcomes can be poor.

    • lemieux2016L says:

      Agree that effective PBL and discovery learning in Science education can be effective if teachers focus on development of skills and conceptual knowledge. Unfortunately, managing PBL and discovery type learning effectively so that the desired learning occurs can be very challenging and requires expert teacher knowledge-content and pedagogical- in Science. Have experiences with both effective and ineffective PBL in Science and and Literacy through a social justice project theme. Strongly concur that while student engagement can be higher, that alone does not result in better learning without all the other key teaching factors being in place before and throughout the process.

  3. Iain Murphy says:

    Was worth reading the linked article because it doesn’t paint a strong picture for PBL. Neither did it paint the negative one that this article suggests. It seems to be negative on the effectiveness of using the results to say anything (which it seems the pro and cons groups for PBL will both do) but otherwise seems rather unexciting.

    It was positive on aspects that seem to be strengths of PBL; oral communication, teamwork, self-directed study. Skills that pro-PBL people might say are important, certainly those in industry do.

    It seemed negative on improving tests scores, which I’m sure the politicians and like who feel that this is the measure of education will find handy.

    I though it was interesting that a lot of schools pulled out making the data questionable. I wonder if this was due to funding, time commitment, community input or student disengagement. If it was the later was that because the program required more consistent work? This is often the case with teamwork where peer pressure is a far greater motivator than the extrinistic demand of test scores.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s