Is book sampling valid?Posted: November 26, 2015
I recently read a post in which a senior member of staff described the process of book sampling in his school. Ideas like this spread like wildfire, especially when broadcast via the internet and by someone of such influence. Soon, everyone might be doing it.
It is worth noting that the blog comes from the UK where the pressures are different than other countries. The schools’ inspectorate there, OFSTED, sets the agenda for schools. Even if OFSTED do not ask for a certain practice, schools may adopt it, thinking that OFSTED would approve of it or under the mistaken impression that OFSTED have mandated it. This is perhaps exacerbated by the tendency for OFSTED inspectors to work with schools as consultants.
However, OFSTED can never be a good reason to adopt a particular approach. The only justifiable reason would be that it improves the quality of education; that it enhances learning. Book sampling as described seems to place onerous marking expectations on teachers and so there would need to be a clear benefit to it.
Research versus accountability
One key problem with book sampling is that a good research tool may not operate well as an accountability tool. For instance, imagine we conducted some research and found that the best teachers tended to spend more time standing at the backs of their classrooms. This observation might have some validity in helping to identify the best teachers. However, once teachers knew that managers were looking for this then even the weakest ones would start standing at the backs of their classrooms when being observed. This is a version of Goodhart’s law which is often paraphrased as:
“When a measure becomes a target it ceases to function well as a measure”.
We may protest that we are offering helpful, low-stakes feedback rather than holding teachers to account. Yet it is hard to interpret critical comments from a senior member of staff in any other way.
A poor proxy
It also seems to me that the quality of marking in an exercise book is a poor proxy for what students have actually learnt. This is even more pronounced in subjects with a large practical component such as PE or Drama because book work is only relevant to a part of the intended curriculum. There is research that the quality of feedback influences learning but this isn’t the same thing at all.
What if we conducted a sample and found a correlation between the teachers whose classes gained the most on external measures and our judgement of the quality of marking? Would this be convincing? Well, it would be circumstantial evidence but it would not imply cause. Perhaps the more effective teachers are better organised and therefore better able to implement the policy? The cause would then be teacher organisation and marking and learning would be two different effects of this. Improving the quality of a particular teacher’s marking would have little to no impact on student learning.What is good marking?
Up to this point, we have assumed that we know what high quality book marking looks like. Usually, this is defined in terms of adherence to a school marking policy. Yet is it hard to trace a through-line from good educational research to the codes, multi-coloured pens, boxes and formalisms adopted by many schools.
If you consider what good quality book marking should achieve then you would perhaps arrive at a list like this:
- Provides useful feedback to the student
- Provides useful feedback to the teacher
- Is positively affective – makes the student feel that his or her work is valued
There are much more focused ways of providing feedback to both students and teachers than through the ritual of taking-up and writing in exercise books. You can use self and peer assessment in class before the lesson has finished. You can use a hinge point question in the middle of the lesson and reteach a concept if it is misunderstood. You can ask a question and get students to write their answers on a mini-whiteboard, immediately assessing what they can do and absolving the teacher of the task of writing the same thing in 30 books. If you want to assess an essay then get students to do this on paper and just take that up like you would for a test.
Although a small delay in feedback might sometimes be beneficial, I am aware of no studies that demonstrate the effectiveness of a smorgasbord of feedback that is given two weeks after the fact when the work has moved on to something else entirely. I doubt this sort of thing has much of an effect even if you ritually require the students to write a prose response. Yes, we should perhaps regularly revisit concepts but this would be better done by a recap or a quiz in class.
Homework is a particularly invalid thing to assess because we have no way of controlling the conditions under which it was done. Middle class students may get help and support from their parents that disadvantaged students don’t receive. Homework assessments would therefore reinforce a bias against the disadvantaged; a bias already identified in teacher assessment.
However, if all we wish is for our students to feel that their work is valued then we can periodically take-up exercise books to just read and sign. We can show we value homework by simply checking that it has been completed (perhaps at the start of the lesson) or by using the products of that homework in the following lesson.
A lazy teachers’ charter?
You might be feeling a little uncertain at this point. If we let teachers off-the-hook on the whole-school marking policy then won’t they just be lounging around drinking instant coffee and eating biscuits all day?
Book marking probably has a very low impact. Effective feedback should be specific and targeted rather than a diffuse mass of colours and codes imposed across subject boundaries. Freed from onerous marking policies, teachers could perhaps adopt more regular in-class assessment that focuses on the particular issues identified within that subject. They could spend more time analysing this data and looking for between-class differences that might identify more and less effective ways of teaching a concept. They could spend time feeding this quality information back into lesson and unit planning.
I reckon that would be better.