A few days ago, I wrote a post about some courses on Project Based Learning that were being delivered to teachers in New South Wales. The promotional material claimed that projects would deliver ‘deeper’ learning. I questioned the fact that no evidence was presented to support this claim and whether we should therefore be spending money on this kind of training. This is a perfectly legitimate thing to question in a free society in which critical thinking is valued.
Sadly, my post has prompted an ill-judged and quite unpleasant personal attack by one of the people involved in the training, David Price OBE. He writes about the fact that I work in an independent school and that I used to post pseudonymously as well as taking a swipe at my book sales! Clearly, he has been doing a bit of research. I have been quite upfront about all of this. Price’s article is a textbook example of the ad hominem fallacy because such attacks bear no relevance to the question of whether there is evidence for deeper learning through projects. If these comments were all the post contained then I would leave it there because I’m obviously not going to respond to this kind of stuff.
[You may purchase a copy of Ouroboros here]
However, Price’s article also addresses the issue of empirical evidence and so this is worth tackling for two reasons. Firstly, he claims the following:
“Mr Ashman says that there’s very little evidence of project based learning’s effectiveness quoted on my website. That’s true. That’s because I believe that busy teachers don’t have time to plough through footnotes, references and sidebars.”
If this is true about teachers then I think it needs to change. If we are to be taken seriously as a profession – a profession capable of leading itself – then we need to be much more evidence savvy. Otherwise, we will continue to be the playthings of those from outside teaching.
And there are alternatives to simply stating references in sidebars. Dan Willingham, for instance, is excellent at briefly summarising a research study to give readers a better sense of the evidence behind the claims that he makes. This is what I try to do.
Project Based Learning Studies
At the end of his post, Price selects seven studies to list, presumably as evidence for Project Based Learning (PBL). I will deal with these in turn because I think they illustrate something important about education research.
The first study is from the American Education Research Journal and is titled, “Beyond Breadth Speed-Test: Toward Deeper Knowing and Engagement in an Advanced Placement Course.” The students taking part in this are high-achieving Advanced Placement (AP) government and politics students. The trial has a comparison group but it is not randomised. Students in the PBL condition come from two schools with generally higher prior achievement (92.7% and 78.8% on a reading/English test versus 77% and 74% for the comparison group) and better social circumstances (11.6% and 32.7% eligible for free or reduced-price lunch compared to 49.8% and 47.8% in the comparison groups). The researchers try to statistically control for the higher prior attainment of the students in one of the experimental schools.
On the AP test, students in the PBL condition performed slightly better than in the comparison schools, with those in the higher achieving school gaining the most. On a measure of deeper learning, there was a positive effect for the high prior attainment school. However, the lower prior attainment PBL school, “did not perform significantly differently than students in the traditional courses.” Which is hardly overwhelming and makes me question the adjustment used for the first school.
The second study is a randomised controlled trial conducted by SRI Education into PBL in science education. I can’t find it on the ERIC database or in my university library; my usual methods of finding out if a study has been peer reviewed. Again, students in the experimental groups had higher prior attainment (this time in science) than those in the comparison groups, making it hard to interpret the moderate effect sizes of PBL which were similar in size to the differences in prior attainment.
The third study is of two fifth grade social studies classes in Turkey. This was not randomised. One class had PBL and one did not. This is a very odd paper because, despite saying that they intend to compute a p-value, they never state what they find or report any quantitative results at all. Instead, we are left to ponder the claim that, “The results showed that there was a significant correlation between the academic successes of experimental and control groups.”
The fourth study looks at high achieving technology students in Israel. The PBL students are from comprehensive schools and the comparison students are from technical schools. It’s clearly not randomised. The students in the two conditions also appear to study different topics. It is therefore hard to tell what any of this means because the authors have varied multiple factors.
In the fifth study, low SES students were given PBL with the goal of achieving the same standards as high SES students. Only a small number of the students ended up being assessed. There was no comparison group of low-SES students receiving a different form of instruction so, although the students made progress in PBL, we don’t know whether it would have been better or worse than if a different approach had been used.
The sixth study of Greek primary school students also appears to have no comparison group and so again makes it difficult to draw conclusions. The seventh paper, a ‘phenomenological study’, again seems to lack a comparison group but I can only find the abstract and it’s not listed in ERIC.
Evidence for Project Based Learning
In my most recent post on this issue, I investigated the evidence of effectiveness put forward by Edutopia. This had the difficulty that it related to something subtly different: Problem Based Learning.
The seven studies listed above at least address Project Based Learning and no doubt have something to add to the PBL debate. However, if they represent the best evidence available for Project Based Learning then I think we have a real problem. In my view, this research is still at an early and experimental stage and is certainly not sufficient to justify spending money training teachers to deliver Project Based Learning at scale. These papers do nothing to allay my fears about equity and I hope that more teachers and administrators become aware of this evidence base and start asking questions.