Education Endowment Foundation to throw money at Philosophy for Children

The Education Endowment Foundation (EEF) was set up in 2011 in England with a grant of £125 million from the Department for Education. They have just announced that they intend to spend £1.2 million on a large scale, 200-school trial of “Philosophy for Children” (P4C) – a program that replaces one literacy lesson per week with a ‘philosophy’ lesson:

“The programme consists of a one-hour session each week, facilitated by the classroom teacher, in which children discuss an interesting philosophical question. Example questions might be ‘Is it fair to have a winner?’ or ‘Is it ok for children to hit teddy bears?’”

This isn’t the worst idea in the world. Many children probably have too many literacy lessons at the moment; lessons where they redundantly practice reading comprehension ‘tricks‘. However, I would prefer that they were replaced with proper subjects like history or geography that have more of a chance of growing a child’s background knowledge.

Why have the EEF decided to spend such a large sum of public money on such an extraordinary venture? Well, they have already run a smaller trial and they claim:

“The previous EEF efficacy trial showed that children taking part in P4C made an additional two months’ progress compared to pupils receiving ‘business-as-usual’ classroom teaching in reading and maths. The trial was robust, with the results being seen on Key Stage 2 assessments and the trial receiving an evidence strength rating of three padlocks. This project will now test the programme in more schools and over a longer timeframe, providing a more robust estimate of the impact.”

Just think about that for a moment. The study found that a program where children are asked to reflect upon their moral responsibility towards cuddly toys improves not only their reading scores but their maths scores too! What astonishing news! Such far transfer effects are as rare as hens’ teeth in education, with one notable exception that I will return to later.

But perhaps it is a bit early to go reconsidering your position on the health of Elvis or whether flying saucers are about to land in the back garden. This previous trial does not yet seem to have been published in a peer-reviewed journal. A number of bloggers with expertise in the area of randomised controlled trials (RCTs) have taken a look at it and come to a quite different conclusion. For instance, Jim Thornton, a professor of obstetrics and gynaecology who is used to interpreting results from RCTs, calls the trial ‘misleading’ and concludes:

“The triallists pre-specified two primary outcomes but only reported one, which showed no difference. They pre-specified seven secondary outcomes which showed no differences either. However when they altered their analysis plan after seeing the data they noticed that two of the secondary outcomes showed a tiny shift in mean change scores favouring the intervention. The effect size was about 10% of a standard deviation, and less than half the participants had the relevant scores measured, but who cares! Without any tests of statistical significance they declared that it was unlikely to have occurred by chance!”

The point about there being no test of statistical significance is an important one. Stephen Gorard, the leader of the group commissioned to analyse the study, does not believe in them. There is an interesting argument to be had about this but the conventional view is that tests such as these tell us something useful. Surely Gorard’s personal opinions are irrelevant here – if the study is funded with public money then it is a public good and as long as there is a public interest in such a test then it should be reported. Gorard can add a footnote disavowing it if he likes.

Conventionally, if a test is not statistically significant then we assume that we haven’t proven anything. With this study, we simply don’t know. Couple this with the retrospective data-mining and it hardly seems like a good basis for spending over a million pounds.

I think I know why the EEF have found themselves in this situation. When I recently saw Jonathan Sharples, head of the EEF, speak, he was full of the benefits of ‘meta-cognitive strategies’. This is an excessively broad category of teaching interventions aimed at increasing thinking about learning. Reading comprehension tricks fall into this group and such strategies seem ripe for expectation effects (e.g. the placebo effect).

Perhaps this has led to an unconscious bias at EEF headquarters. During Sharples’ presentation he suggested that the results from trials of meta-cognitive strategies were consistently positive with a similar effect size. Yet he seems to have forgotten about the recent EEF trial of cognitive acceleration, a meta-cognitive approach to science lessons. This trial found no effect for cognitive acceleration.

And the history of cognitive acceleration is instructive. The original studies were conducted in the late 80s and early 90s. They caused quite a stir at the time because they demonstrated far transfer effects – it was claimed that a science intervention led to improvements in English Language results several years later.

Does that remind you of anything?

Correction: I asked Stephen Gorard and he said that a peer-reviewed paper had been published although he could not link me to it. I’ve now found this paper. It’s in the Journal of Philosophy of Education and is available on ResearchGate.

Advertisements

12 Comments on “Education Endowment Foundation to throw money at Philosophy for Children”

  1. Queen's English says:

    Reminds me of some dodgy PD I had the other day.

  2. Dylan Wiliam says:

    A bit of wider context might be useful here. While the previous EEF trial produced results that were difficult to interpret, there is at least prima facie evidence that Philosophy for Children can have some effect on student achievement:

    Topping, K. J., & Trickey, S. (2007). Collaborative philosophical enquiry for school children: cognitive effects at 10-12 years. British Journal of Educational Psychology, 77(2), 271-288.

    Trickey, S., & Topping, K. J. (2004). ‘Philosophy for children’: a systematic review. Research Papers in Education, 19(3), 365-380. doi: 10.1080/0267152042000248016.

    Given that positive effects have been found in a number of trials, and, in particular that the review by Trickey and Topping found that the effects didn’t vary much from study to study, I think Philosophy for Children is very different from Cognitive Acceleration in Science Education, which has only one major study showing substantial impact on student achievement.

    And while I agree that it seems unlikely that a science intervention increases student achievement in English because of “cognitive acceleration”, if the mechanism is rather that the intervention helps students believe that thinking helps when you’re stuck (in other words, a change in beliefs rather than executive function) then the far transfer makes more sense.

    Given the potential benefits, and the low cost of implementing Philosophy for Children, I think spending £1.25m makes sense.

    • gregashman says:

      I think we are going to have to disagree here. The justification given by EEF is based upon their previous study which simply doesn’t stack up. I’ll have a look at the other studies to see if they are well designed. I am not as convinced that there is a plausible mechanism here as you are.

    • Janita says:

      I was interested in Dylan Wiliam’s mention of “a change in beliefs rather than executive function”. Is it possible that people cling to their faith in the notion of transferable skills despite the evidence against it because they confuse skills with attitudes and habits? As for the “philosophy” that is proposed, it looks very much like another invitation to children to opine rather than think. I see no mention of any lessons in basic logic, sadly.

      • Janita says:

        I can’t cite any studies to support this, but I would have thought that a philosophy course for kids would be useful if it meant teaching them about the ideas of humanity’s great thinkers and the elements of straight and crooked thinking.

      • I support Greg’s argument with respect to the inadequacy of the EEF research, but I think the general presumption against transferable skills has not been properly thought through – see my discussion with David Didau at http://www.learningspy.co.uk/learning/robots-evolution-schools-shouldnt-teach-innate-skills/#comment-71742. To my mind, Dylan’s reasoning with respect to getting into the habit of thinking, and the likelihood that such a habit might be transferred from one domain to another, is entirely plausible… and it is not because I automatically agree with everything that Dylan says!

        Second, I challenge the idea that there *is* a substantive difference between skills and attitudes. Take the logical positivism challenge, which asks “how do you know?” https://edtechnow.net/2016/09/20/elephant/ And the answer in both cases is “by observing what the student can do”. That is why I think we want to move beyond all these unhelpful arguments about knowledge vs skill vs attidudes, which are about as useful as the filioque debate about the exact relationships between the Christian Father, Son and Holy Ghost. Much better that we should just talk about capability, something that can be defined explicitly and makes no such meaningless distinctions about unobservable phenomena.

        Crispin.

  3. […] Education Endowment Foundation to throw money at Philosophy for Children → […]

  4. […] EEF is as prone to prejudice as anyone. Take the example of the decision to fund further research in Philosophy for Children. There are serious misgivings with the way the trials so far have been run and Greg Ashman makes […]

  5. […] report you see that it includes writing interventions alongside critical thinking interventions, philosophy for children, interventions that seem to target executive function, cognitive acceleration based on the old CASE […]

  6. […] Foundation in England would pay more attention to mechanisms. If so, they might pause before throwing even more money at Philosophy for Children, a distal intervention that is intended to improve English and […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s