The Emperor’s New Project

A recent study found something quite counterintuitive. Girls who were given an infant simulator were more likely than those who had a standard health education program to become pregnant in the following years. The idea of the simulators was to give girls a sense of how difficult it is to care for a baby and therefore reduce the chance of pregnancy.

This is not the final word in the matter. There could be specific reasons why this particular program failed and others might work better. But it’s worth knowing. 

I’m pretty sure all of those involved with the infant simulator would have thought it was a great program. They would probably have told you how engaging the girls found it and how they had all agreed that looking after the baby was such hard work. This is why we need experimental data in the social sphere. We cannot trust out intuitions; our personal experiences. Those who argue that areas like education need a lower standard of evidence than, say, medicine are wrong.

I have recently been exploring the evidence for Project Based Learning because of the claim that it delivers ‘deeper’ learning. The evidence base for this is extremely weak. One commenter pointed me towards Edutopia and its papers related to Problem Based Learning. I also explored some other studies that has been cited to promote Project Based Learning. Even with weak study designs, many of them don’t really support the claim that projects deliver anything superior. On Twitter, Corinne Campbell recently pointed to the hierarchy of evidence that the New South Wales Centre for Education Statistics and Evaluation uses to evaluate professional development. Most of the Project Based Learning evidence would struggle to reach silver standard. 

Perhaps this is because Project Based Learning is a new approach? After all, it is associated with the 21st century skills movement. Perhaps there has not been enough time to research it? Is it then right to keep this promising approach from children while we wait for this evidence?

Project Based Learning may be many things but it’s not new. John Dewey was using it in his laboratory school back at the start of the 20th century and William Heard Kilpatrick wrote an influential essay on the topic back in 1918. If you strip out the sexist assumptions and old-fashioned language, Kilpatrick’s essay resonates with what people say about projects today:

“Suppose a girl has made a dress. If she did in hearty fashion purpose to make the dress, if she planned it, if she made it herself, then I should say the instance is that of a typical project. We have in it a wholehearted purposeful act carried on amid social surroundings. That the dressmaking was purposeful is clear; and the purpose once formed dominated each succeeding step in the process and gave unity to the whole. That the girl was wholehearted in the work was assured in the illustration. That the activity proceeded in a social environment is clear; other girls at least are to see the dress.”

So if there is evidence for Project Based Learning then we should have it by now. Educationalists having been promoting the idea for a long time. So why is the evidence not forthcoming? 

I think this is because people mistake activity for learning. In a project there is lots to do. But much of this activity is inconsequential: things like looking for the scissors. This makes projects inefficient and it draws students’ attention to the wrong things.

One of my PhD supervisors did an experiment in the 1980s. Undergraduates were given a series of problems. Each problem involved a starting number and a goal number. The participants had to get from the first number to the second using only two moves which they could repeat: multiply by three or subtract 29. The problems were designed so that each one was solved by alternating the steps. Although the students could generally solve the problems, very few ever worked out the rule.

This is because the process of attending to the solving of the problem took up so much attention that little was left to notice patterns. This is likely to be what happens with projects. There is too much extraneous information to filter and so little is learnt.

Which is why a teacher guided approach where students’ attention is drawn to key features is so effective.

Am I calling for a ban on projects? No. They can be effective once students have the right knowledge and skills. They may also be less effective but still worth doing with novice learners just to vary a day’s activities and do something a bit different. I am fine with that. I just don’t think people should be claiming that projects are an effective way to learn and I certainly don’t think we should be investing professional development funds in training teachers to deliver projects at scale. Not if we believe in an evidence-based approach.

Advertisements

19 Comments on “The Emperor’s New Project”

  1. I’m not sure you have managed to make the case here that teacher guided and projects are incompatible? Could a project also be teacher guided? And would that remove some of your objections to project based learning? Also if we accept that project based learning can be teacher guided with the teacher perhaps moving to the role of coach as appropriate expertise is developed, then project based learning done this way will essentially be a form of deliberate practice. Would that make it more palatable?

    In terms of evidence for project based learning being weak, I’m assuming then that you’ve also rejected arguments for project competitions like the Google Science Fair or National History Day? NHD did publish some supporting research in 2011 for their approach here https://nhd.org/sites/default/files/whynhdmatters/NHDReport_Final3.pdf with the executive summary here https://nhd.org/sites/default/files/whynhdmatters/NHDExecutiveSummaryfinal.pdf.

    • gregashman says:

      Fully guided instruction is compatible with projects if you first fully and explicitly teach all of the required knowledge and skills. Hattie has a YouTube video on this. I wouldn’t call this ‘project-based’ however.

      I’m not sure what a fully guided project would look like. If a teacher is giving explicit instructions in each step as the students go through them – “take this and place it there” etc – then that doesn’t strike me as a project.

      Projects don’t meet the definition of deliberate practice because deliberate practice involved isolating specific skills and training those, whereas projects focus on a complex performance of integrated skills and knowledge.

      I will take a look at your links.

      • Your definition of deliberate practice is and interesting but perhaps a partial one, certainly if compared to how Ericcson describes it in Peak. Ericcson describes isolating specific skills and training those to improve overall performance. He then identifies the underlying mechanism of expertise as the relative complexity of the underlying mental patterns in the performer (ie not specific skills in isolation). This seems to be very similar to what a teacher guided project would look like? Eg: Teacher giving explicit instructions initially and as the students developed in expertise the teacher’s role would adjust slightly to constructing and giving specific drills for specific aspects of the student’s project.

      • gregashman says:

        No. That’s not my understanding of deliberate practice. This would be eg a golfer alternating between practicing a drive and a pitch. PBL would be like playing a game of golf. And I still don’t see how something that is fully guided could be described as a project.

  2. […] load theory, I came across something weird that I couldn’t explain. A paragraph from Greg Ashman’s latest reminds me of this puzzle. It’s really small and inconsequential, but it’s been bugging […]

  3. It doesn’t seem to let me reply to your comment? I think it would be more like a golfer working on his drive in isolation and then putting it back into his overall game. I think you’re assuming that the level of guidance would remain constant throughout the project. Wouldn’t it be more at the start and then decrease (or at least change in form) as expertise developed?

    • Chester Draws says:

      as expertise developed

      For secondary school students how far do you think we would get down the road towards expertise in the time we have?

      Certainly English teachers show their students how to read and analyse a book for years on end without the vast bulk of students coming close to being able to do that by themselves.

      Expertise doesn’t just require going through the actions several times. It requires an intellectual maturity most high school students simply don’t have.

      They might be able to do a Social Studies project by the time they are 14, having done several a year for many years. But they couldn’t necessarily do a decent History project as a result, because their expertise is extremely limited in scope. Which is why History teachers have to teach their subject from the ground up, even though the students have many years of practice at Social Studies.

    • Iain Murphy says:

      Ryan until “mastery” is achieved deliberate practice methods are always very hands on between teacher and learner (better description would be mentor and protégée) as the teacher is reflecting on what skills the learner has developed and ways to develop.

      My concern is this happens very rarely in a class setting. The chances of hitting the deliberate practice required for students with a huge range of skills both in the topic and more generally in learning can’t happen effectively on a large scale. This is best done one to one.

  4. Iain Murphy says:

    Greg as you write your ideas are developing which is great to see.

    My concern with both project or teacher guided is motivation and engagement of the student. I’ve just sat through a meeting were both techniques were exonerated for their merits and by the same person lamented for having disengaged students. Assuming that students care because the teacher cares is a recipe for disaster. Worse is when the task has been mandated.

    I’ve run great projects that often required teacher guided sessions as students discovered the need for a skill. At that point becoming explicit in what is needed works very well.

    I’ve seen failed projects as the students didn’t have the skills or context to achieve, mostly the cases Greg uses to lament project based learning and I completely agree with him.

    I’ve also seen what happens when teacher guided becomes the only style -coughs- maths -coughs- and students become disengaged as they only ever develop skills without ever getting to use them with a lot of practice and little deliberate practice so growth mindset is lost.

    I’m starting to wonder if the sweet spot involves technology around strong projects that engage students (high tech high model maybe?) where the technology can help communicate the skills (YouTube for lectures, Skype for one-to-one, better assessment tools please) and allow experts to mentor for better deliberate practice while others can help guide the workflow (developing planning skills, group work, studying, etc). However I feel I’m getting off-topic and should start my own blog to develop this idea into something.

    Greg, you keep making me think. Please keep the great posts coming.

  5. Stan says:

    Greg may be missing the important benefits of PBL.

    One of the most important lessons about collaboration is the value of the critical step in picking your collaborators. PBL can offer an invaluable life lesson here.

    Some students will learn that there are social loafers who when the reward is shared making the minimum contribution you can get away with is a winning move and can make school less of a chore.
    Other students will learn that there exist social loafers and team projects can make school far less enjoyable than it could be.

    Other students will learn they can get good marks on a team project in science without learning any science by being good at artwork and page layout. The lesson here for those that don’t like science is the less science and more finding the scissors there is the more they will enjoy science class.

    Now these lessons are like burning your fingers it only takes one lesson to discover all you need to know and no amount of instruction will do that as well as actually burning your fingers.

    Of course Greg is not arguing that there should be no projects he is arguing about the relative efficiency for learning the material. The benefits of one science fair project a year may be many in terms of getting some students excited about what they have learned and those important collaboration lessons. Kids who are good at science do tend to enjoy getting involved in science fair projects. So perhaps this is a bit like letting kids who enjoy a sport actually play a game rather than just practice.

    I would argue math is different. There are no math fairs with math projects on display. People who enjoy mathematics tend to do math competitions or mathematics outside of the school curriculum. These may include teams but the work is predominantly individual effort. Students who enjoy recreational mathematics are more likely to spend time on one of Ian Stewart’s books than working on a project. Korner’s The Pleasures of Counting is my exemplar of an interesting math book but most of that is only available to students close to the end of high school. Maybe my perspective is too narrow and there are cases where students are motivated about math through project work.

  6. Brian says:

    I think I would have to agree with much of what was said on both sides.

    This all resembles the joke my dad used to tell me when I was a nipper…..”two boys went into the greengrocer and one took and apple, gave it to the other who hid said apple behind his back”…..the one who took it said “I haven’t got it” and the one who has it says “I din’t take it” and of course eveyone is telling the truth.

    For this discussion to be of any value, we would have to define clearly what we mean as “project based learning (PBL)”. I have my own definition which is about learning by engaging in projects. My definition would make no prescriptive reference to minimally or maximally guided as PBL refers to the task/activity.

    The debate seems to have bolied down here to the old chestnut…..”minimally guided vc direct instruction” but for me the issue is irrelevant. PBL for me will likely involve some direct instruction, some minimally guided instruction and all sorts of alternatives in between.

    The subject area and learning outcomes will affect the most effective methods of instruction and dictate whether in fact PBL is appropriate.

    Depending on ones views on the purpose(s) and nature of education and the educational process, the issue of PBL morphs into different types of issues.

    Arguing that DI is better than PBL is a bit like arguing that cats are better than dogs.

    • gregashman says:

      I don’t agree. What would a fully guided project look like? A teacher would have to decide the goal and all of the steps to achieve that goal. She would have to instruct the students in each step. This doesn’t look much like a typical project. Even then, much of the students’ focus will be on the doing – “take this and put it there” – rather than on any learning objectives.

      In reality, of course, projects are not fully guided and so they imply student choice and students finding some things out for themselves. Therefore they *are* in opposition to a fully guided approach.

  7. […] I want to make is that completing a task is not the same thing as learning from it (see the ‘multiply by three and add 29‘ experiment’). If your brain is totally occupied with doing something then there […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s