When project-based learning works

Embed from Getty Images

We all know the story of project-based learning. It has been around since at least 1918 when William Heard Kilpatrick wrote The Project Method: The use of the Purposeful Act in the Educative Process and yet, over all this time, it has accrued very little evidence of effectiveness. Perhaps the most significant recent development was a randomised controlled trial run by the Education Endowment Foundation (EEF) in the UK. This was necessary because, as the EEF state in their literature review, “The existing evidence for a causal link between PBL and attainment outcomes seems to be weak. Most of the reviewed studies did not involve random allocation of participants to control and experimental groups and, as a result, a causal link between project based learning instruction and positive student outcomes has not been established.” It is quite extraordinary to reflect that we are in this position given the long history of project-based learning and its current popularity, but there we are.

In their trial, the EEF found, “Adopting PBL had no clear impact on either literacy (as measured by the Progress in English assessment) or student engagement with school and learning.” They also found a possible negative impact on the literacy of students eligible for free school meals. However both findings have to be treated with caution because so many schools dropped-out of implementing project-based learning during the study. It is due to this background that I predict that project-based learning will soon reappear under a new name – watch this space.

However, I did recently come across a study that demonstrates a positive effect of project-based learning on scientific understanding which I thought was worth investigating. Tara Craig and Jill Marshall, the authors of the new study, agree with the EEF that, “…there is a lack of studies randomly assigning students to receive PBL,” and so set-out to design a study of their own. This sounds promising – random assignment is the best way of determining a cause-and-effect relationship between a teaching approach and its effects on learning.

Once I read the body of the paper, however, my optimism faded. Students were not randomly assigned to project-based learning or some other approach within once school or a group of schools. Instead, entry to Manor New Technology High School, a school using project-based learning, was determined on a lottery basis, with the losers going to the more conventional Manor High School and not receiving project-based learning. These formed the intervention and control groups.

Lotteries are often used for entry into Charter schools in the US and this approach has been used to attempt to determine the effects of Charter schools. It is therefore clear that project-based learning is not the only factor that varies between the groups and the effectiveness of the school would also contribute. It is possible, perhaps likely, that Manor New Technology School attracts better or more motivated teachers and there is also likely to be an effect on the students of gaining entry. I suggest this is likely because Manor High School does not sound great. It had a lower school performance ranking and teacher recruitment and retention issues. For this reason, professional development was focused on classroom management rather than project-based learning.

Despite these clear differences, when ethnicity and economic disadvantage are controlled, there was no significant difference between the intervention and control group on four out of five measures involving maths and science performance. However, there was a significant difference favouring the intervention in Year 10 Science.

This is hardly encouraging evidence in support of project-based learning.

 

Advertisements

7 thoughts on “When project-based learning works

  1. If a school’s CPD is still focused on classroom management, surely this is a pretty good indication that teaching and learning is poor. PBL may well not be the most significant variable involved in this study. In fact, the difficulty of identifying and quantifying all possible variables that might affect results is why I’ve become a tad sceptical about the value of RCT.

    1. Surely that is not an RCT problem but a problem of properly matching the experiment to the resources available.
      If you tried to measure some small difference – PBL with teams of 2 verses individuals doing a project by themselves then you wouldn’t need as much data as if you are comparing PBL to any alternative (ie a PBL verses whatever else is done).
      The problem is the ambition to make a big claim gets in the way of the ambition to make a robust correct claim.

  2. When you say one of the five measures showed a significant improvement for the intervention does that mean they took into account they were measuring five things when setting s criteria for significance?
    My point is if you measured 20 things and only one showed a significant result you could claim nothing with the normal significance criteria for a single measure.

  3. This time is very misleading. I’ve learned nothing about when pbl works. Are you advocating for or against it??

  4. My personal experience in 27 years of teaching in public schools says that PBL not only has a positive effect on my students academically but also behaviorally. There is a lot to be said, as well, for differentiation. Some students do better with a book in front of them and a pen and paper. We need to focus on being sure our children know how to learn and that they have a well rounded set of tools to move on.

    1. You should write to Hattie Mimi and make sure he knows about this. Evidence be damned, we’ve got some high quality anecdote.

      My personal experience is that students behave poorly whenever they don’t know what they are supposed to be doing. And nothing gets them into such a state quicker than not knowing where to start in a project or other open-ended situation. So when I have them do projects, I guide them quite explicitly through the process. Which is exactly the opposite of PBL, of course.

  5. Generally one way of getting a positive result is that the worse half drop out. That only leaves the ones who are succeeding, and so a positive result can therefore arise out of nothing. If the half of the schools that were finding it hardest dropped out of this program, then you should expect a positive result if the program was exactly as good as normal teaching. Tack on any Hawthorne effect, a bit of p-hacking etc, and any program should get some effect.

    Five schools dropping out in a study that only lasts a year is outrageously bad, especially when they are getting support to run the trial program. They can’t all have had senior leadership changes — and why would such changes make any difference unless the management were reacting to staff concerns about the trial?

    So the report line “It may be that successful implementation of the PBL programme is difficult for schools in such circumstances” is thus amusingly positive spin on a terrible result. Because I don’t see much evidence of any “successful” implementation at all.

    The much more obvious conclusion is that PBL is far worse than what the schools were doing before.

    The standards of evidence in educational research are appallingly low. Imagine if a Head of Department excluded the worst 40% of grades for one teacher but not for the others and then reported to the principal that they were all as good as each other. Or that one principal said that his school was as good as another, despite a drop-out rate of 40% compared to 10%, because the remaining students did equally well. Just to do so would be laughable. Yet apparently we are meant to swallow the equivalent from the EEF.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.