I have been critical of organisations such as the Education Endowment Foundation in the UK. One of my criticisms has been the sole focus on large randomised controlled trials, randomised at the school level and therefore often under-powered (i.e. lacking a sufficient number of participants to produce a clear result).
I think we should focus as much on shorter, smaller-scale randomised controlled trials that are randomised at the student level. Instead of attempting to measure the effectiveness of whole packages of intervention, they could measure simple single-factor changes to teaching. You might wonder about whether such trials can answer questions useful to classroom teachers. I would argue that they can.
Imagine, for instance, a trial to attempt to measure the difference between receiving written feedback on an essay as opposed to whole-class feedback. This is a key issue because many school marking policies insist on written feedback and yet the evidence to support this practice is weak. Some schools have started delivering whole-class feedback instead.
Here’s what you could do to test these approaches:
- Ask two classes of around 50 students in total to complete an essay in a lecture theatre during an English lesson. Generate a unique number for each student and place this on their script rather than a name.
- By number, randomly assign 25 of these students to Condition A and 25 to Condition B.
- For Condition A students, provide written feedback on their script.
- For Condition B students, do not provide written feedback.
- At the start of the next lesson, Condition A students go to a classroom where they have 15 minutes to read their written feedback with no teacher instruction. In a different classroom, Condition B students receive whole-class feedback from the teacher who marked the essays.
- Students then go back to their usual non-randomised English lessons for the remainder of the period to be taught something that is not closely aligned to the essay work.
- At a later date, ask the students to write a follow-up essay that is similar to the original one, again annotating each script with the same student number as in the first essay.
- Mark these essays using comparative judgement.
- Analyse the data to see if there is a statistically significant difference between students in Condition A and students in Condition B.
No single test of this kind would be definitive because there is endless scope to argue over the details. However, if the same pattern emerged after several replications, we might be able to draw a clear conclusion.
This would be of practical value to English teachers. It would not necessarily force teachers to give written feedback or whole-class feedback. Some teacher may choose to do both. Some teachers may note a slight advantage for written feedback, if one is found, but conclude that this is not worth the cost in time. Regardless, these decisions would be better informed by education research that is actually useful.
If you are working in a school that is thinking about the value of written feedback, why not partner with a university and do this research?