An open book?

Embed from Getty Images

I teach two Year 12 classes that have an external exam at the end of the year. One is the Victorian Certificate of Education (VCE) exam in Physics and the other is VCE Mathematical Methods. Despite all the claims made about standardised testing in Australia, the VCE and its equivalents in other states, are really the only high stakes tests that students sit. They are high stakes because entrance to university can depend upon the results. For that reason, I am minded to take a low risk approach.

VCE exams are not the exams that I would design. They have been influenced by constructivist ideas which have worked to both water-down and decohere the curriculum and the way it is assessed. For instance, Maths Methods has two exams. Exam 2 is worth twice as many marks as Exam 1 and students are allowed to bring in one bound A4 sized book of notes and a sophisticated calculator. In Physics, there is only one exam, but students are allowed to bring a sheet of A3 paper with both sides covered in notes – this is popularly known as a ‘cheat sheet’. Furthermore, part of the VCE grade comes from tasks that students complete in class and that are then cohort-referenced against the exam grades. These have lots of regulations around them that, in some cases, include the explicit instruction that students must be able to refer to notes.

I am not a fan of open-book assessment. Firstly, it does not fit with my preferred model of learning which is about maximising long-term memory. Instead, it seems to be based on the idea that mere facts are unimportant and that learning is about developing mysterious skills that are somehow independent of knowledge. On a more prosaic level, I worry that students will be misled. They will be lulled into a false sense of security by thinking they have everything then need to know written down. However, if they haven’t actually learnt the content, they will be plagued by all the problems associated with reading content when you lack background knowledge.

The most transparent example of this that I can think of is the two ‘split ring commutators’ in the Physics syllabus. One is associated with a Motor and the other with a certain kind of generator. When the assessors ask students to explain the role of a split ring commutator in a generator, those who are over-reliant on their cheat sheet are likely to explain the role of a split ring commutator in a motor. Fortunately, I know about this booby trap and warn my students about it in an animated fashion.

However, I don’t quite have the courage of my convictions. I don’t really know what the evidence says about open-book assessments and so I follow a low risk approach by replicating the final exam as much as possible. For instance, in class work and assessments, if I am giving Maths Methods students Exam 2-style questions to complete, I let them refer to their notes and I let my Physics students refer to their cheat sheets throughout the course.

Should I?

Perhaps not. A new study involving university students learning cognitive psychology seems to suggest that students who are not allowed access to their notes perform better overall. Unusually for a trial of this kind, it is not a quasi-experiment but a full randomised controlled trial and so the evidence it provides is pretty strong. It is only one study, so we probably shouldn’t give too much weight to it, but it has made me pause.

Perhaps I need to change my approach.

Standard

6 thoughts on “An open book?

  1. Students in Maths Methods are usually under such time pressure that they can’t afford to rely on the bound book.
    I don’t have a problem with the Physics cheat sheet. It is not a key skill that students commit formula to memory. The study you referenced here does make us question our approach.
    I want them to do the study again but let the students have a cheat sheet in the exam. It is clear that in the cheat sheet vs non cheat sheet grouping the non cheat sheet group will benefit on a non cheat sheet exam. What would happen if the same study was run with a cheat sheet allowed exam?

  2. Tangentially related: in the Latin HSC a few years ago they began allowing the kids to bring in dictionaries for the exam. Practically all the CLTA teachers were against it, but for some reason the wonks thought it was a good idea because “they do it in the IB” (actually that’s not entirely true, but anyway). Predictable result: the course has now had the guts of its linguistic content ripped out of it, and the kids have become far lazier with vocabulary.

    Open-book exams are just a contradiction in terms, in my view.

  3. There is a bigger issue here, even if you improve the long term memory of all students in Year 12, the assessment sytem being a ranking system based on the Normal Distribution, would not measure that improvement. Roughly 50% of students would still get a study score below 30 out of 50 even though their long term memory was maximised.

    • True. But this is the least worst option. The only alternative is criterion referencing which inevitably gets gamed. Look at the rampant grade inflation involving criterion-referenced GCSE grades in the early 2000s in England.

  4. Stan says:

    I asked my son for a students opinion on this. So just an anecdote but he is in 3rd year math and physics at uni to plenty of experience on various exams. Math don’t allow cheat sheets, physics do. His view, in general there is not time to rely on the cheat sheet and you don’t need to. But the exercise of making the cheat sheet is useful.

    As one alternative is an officially provided cheat sheet it would be interesting if any experiment compared the self constructed ones to use of a provided one.
    By his logic my son should be making cheat sheets for his math exams even if he can’t bring them in. There might also be benefits to redoing cheat sheets or entering them straight into a tool like anki for memorization practice.

    For memorization do you introduce your students to ankiweb.net ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.