More evidence for Project Based Learning

A few days ago, I wrote a post about some courses on Project Based Learning that were being delivered to teachers in New South Wales. The promotional material claimed that projects would deliver ‘deeper’ learning. I questioned the fact that no evidence was presented to support this claim and whether we should therefore be spending money on this kind of training. This is a perfectly legitimate thing to question in a free society in which critical thinking is valued.

Unpleasantness 

Sadly, my post has prompted an ill-judged and quite unpleasant personal attack by one of the people involved in the training, David Price OBE. He writes about the fact that I work in an independent school and that I used to post pseudonymously as well as taking a swipe at my book sales! Clearly, he has been doing a bit of research. I have been quite upfront about all of this. Price’s article is a textbook example of the ad hominem fallacy because such attacks bear no relevance to the question of whether there is evidence for deeper learning through projects. If these comments were all the post contained then I would leave it there because I’m obviously not going to respond to this kind of stuff.

price-ouroboros-footnote

[You may purchase a copy of Ouroboros here]

Empirical Evidence

However, Price’s article also addresses the issue of empirical evidence and so this is worth tackling for two reasons. Firstly, he claims the following:

“Mr Ashman says that there’s very little evidence of project based learning’s effectiveness quoted on my website. That’s true. That’s because I believe that busy teachers don’t have time to plough through footnotes, references and sidebars.”

If this is true about teachers then I think it needs to change. If we are to be taken seriously as a profession – a profession capable of leading itself – then we need to be much more evidence savvy. Otherwise, we will continue to be the playthings of those from outside teaching.

And there are alternatives to simply stating references in sidebars. Dan Willingham, for instance, is excellent at briefly summarising a research study to give readers a better sense of the evidence behind the claims that he makes. This is what I try to do.

Project Based Learning Studies

At the end of his post, Price selects seven studies to list, presumably as evidence for Project Based Learning (PBL). I will deal with these in turn because I think they illustrate something important about education research.

The first study is from the American Education Research Journal and is titled, “Beyond Breadth Speed-Test: Toward Deeper Knowing and Engagement in an Advanced Placement Course.” The students taking part in this are high-achieving Advanced Placement (AP) government and politics students. The trial has a comparison group but it is not randomised. Students in the PBL condition come from two schools with generally higher prior achievement (92.7% and 78.8% on a reading/English test versus 77% and 74% for the comparison group) and better social circumstances (11.6% and 32.7% eligible for free or reduced-price lunch compared to 49.8% and 47.8% in the comparison groups). The researchers try to statistically control for the higher prior attainment of the students in one of the experimental schools.

On the AP test, students in the PBL condition performed slightly better than in the comparison schools, with those in the higher achieving school gaining the most. On a measure of deeper learning, there was a positive effect for the high prior attainment school. However, the lower prior attainment PBL school, “did not perform significantly differently than students in the traditional courses.” Which is hardly overwhelming and makes me question the adjustment used for the first school.

The second study is a randomised controlled trial conducted by SRI Education into PBL in science education. I can’t find it on the ERIC database or in my university library; my usual methods of finding out if a study has been peer reviewed. Again, students in the experimental groups had higher prior attainment (this time in science) than those in the comparison groups, making it hard to interpret the moderate effect sizes of PBL which were similar in size to the differences in prior attainment.

The third study is of two fifth grade social studies classes in Turkey. This was not randomised. One class had PBL and one did not. This is a very odd paper because, despite saying that they intend to compute a p-value, they never state what they find or report any quantitative results at all. Instead, we are left to ponder the claim that, “The results showed that there was a significant correlation between the academic successes of experimental and control groups.”

The fourth study looks at high achieving technology students in Israel. The PBL students are from comprehensive schools and the comparison students are from technical schools. It’s clearly not randomised. The students in the two conditions also appear to study different topics. It is therefore hard to tell what any of this means because the authors have varied multiple factors.

In the fifth study, low SES students were given PBL  with the goal of achieving the same standards as high SES students. Only a small number of the students ended up being assessed. There was no comparison group of low-SES students receiving a different form of instruction so, although the students made progress in PBL, we don’t know whether it would have been better or worse than if a different approach had been used.

The sixth study of Greek primary school students also appears to have no comparison group and so again makes it difficult to draw conclusions. The seventh paper, a ‘phenomenological study’, again seems to lack a comparison group but I can only find the abstract and it’s not listed in ERIC.

Evidence for Project Based Learning

In my most recent post on this issue, I investigated the evidence of effectiveness put forward by Edutopia. This had the difficulty that it related to something subtly different: Problem Based Learning.

The seven studies listed above at least address Project Based Learning and no doubt have something to add to the PBL debate. However, if they represent the best evidence available for Project Based Learning then I think we have a real problem. In my view, this research is still at an early and experimental stage and is certainly not sufficient to justify spending money training teachers to deliver Project Based Learning at scale. These papers do nothing to allay my fears about equity and I hope that more teachers and administrators become aware of this evidence base and start asking questions.

Advertisements

9 Comments on “More evidence for Project Based Learning”

  1. Well I’ve got the popcorn out ready for the return serve.

  2. Stan says:

    In Ontario our ministry has some good words to say about PBL.
    http://www.edu.gov.on.ca/eng/literacynumeracy/inspire/research/WW_BestPractices.pdf

    But they do note
    “At its worst, PBL can be perceived by both teachers and students as an inefficient use of time that does not sufficiently address the depth of subject knowledge needed by students.”

    It is worth noting the odd wording here – the implicit claim that this is just a perception issue.

    The ministry report claims “By effectively employing the same techniques that professional project managers use, teachers can circumvent the risks to PBL and help young people acquire the 21st century habits of mind that will be indispensable to them in their lives.”

    If you look at what a PMI course costs you will find it takes months and costs thousands. That would take you to an entry level of expertise in project management. If you were going to acquire these skills as an essential part of your job skills you would want someone with lots of hands on project management experience teaching it not someone with a few hours of seminar time behind them.

    This could drift into a rant about the way some educators at the elementary and high school level seem to ignore the years of study under subject matter experts that some fields required before people are ready to even enter a field.

    • gregashman says:

      Yes. There are plenty of reasons why we might be sceptical of a project based approach and the expertise gap is a pretty fundamental one. This makes robust evidence even more important.

  3. Stan says:

    Another useful link
    http://www.instituteforgovernment.org.uk/sites/default/files/publications/4545%20IFG%20-%20Showing%20your%20workings%20v8b.pdf

    This paper on how to evaluate proposals includes the great term “evidence sprayed” to distinguish poor quality proposals from evidence based ones. Evidence spraying seems to be what you face here – lots of references but all of poor quality.

    As the paper on the link points out one of the main features of a solid proposal is it addresses all the best available counter claims and costs.

    One cost of PBL if done according to project management principles is that the work will be divided up and so no student will cover all the material. That may well be okay but obviously should be a big warning on any pitch for the benefits of PBL.

  4. Tempe says:

    In my opinion it is he who attacks and makes this personal. Is he really threatening to sue? This takes things to a whole new level. Also, I wonder what he meant when he said you “reported” him ( in inverted comma’s) to BOSTES?

    It’s really good to see the evidence for PBL looked at with a critical eye. As you point out it is all rather underwhelming considering the cost and time put into these courses. So much money to be made in Ed.

    • jennifer stephenson says:

      Threatening to sue is a standard tactic of edupreneurs when you question their claims, or suggest their product is not evidence-based. Personal experience. DOn;t worry Greg, a number of us would throw in for your defence.

  5. […] and its papers related to Problem Based Learning. I also explored some other studies that has been cited to promote Project Based Learning. Even with weak study designs, many of them don’t really support the claim that projects […]

  6. […] personal attack from one of the consultants delivering the New South Wales training. There was much sound and fury but little light shed on the key question of the effectiveness of project-based […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s