How to improve Australia’s writing assessments

Embed from Getty Images

It is being widely reported that Les Perelman, a retired academic from the U.S., has analysed the Australian NAPLAN writing assessment and found it wanting. NAPLAN stands for ‘National Assessment Program – Literacy and Numeracy’ and it consists of a suite of tests taken by Australian students in Years 3, 5, 7 and 9. Parents can access school-level results via the MySchool website, raising the stakes for schools, if not the children themselves.

I quite like the numeracy assessment, although I was strongly opposed to the recent move to reduce the size of the non-calculator section in Years 7 and 9. The ‘Language Conventions’ paper that assesses components such as grammar and spelling seems reasonable to me, but the greatest flaw in the system is the two papers in reading and writing. Both of these papers assume that reading and writing are entirely generic skills and assess them accordingly.

This becomes clear when you consider Perelman’s criticism of the writing paper. According to writing style guides, “…one should use the simplest, most precise language wherever possible,” and yet the NAPLAN writing paper awards credit for using big words. I understand what Perelman is saying. I’ve had enough of my writing edited for clarity to know that this usually involves simplification, but that only applies if we are trying to communicate with the greatest clarity possible. Often, I am writing about pretty complex and abstract ideas and so a focus on clarity is key. Much professional writing looks like this, hence the style guides.

But what of art? Isn’t writing sometimes about building up tension or communicating beauty? In such writing, merciless economy is less of a goal. We no longer look upon a ‘stream’ but a ‘silver serpent, slithering across the sun-gilded valley floor.’ If you wrote office emails like that then people would think you were odd, but it’s great for children’s stories.

So what is it? What type of writing is NAPLAN trying to assess? Well, students could be asked to write a narrative based around some kind of prompt such as ‘The Box’ or they could be asked to write persuasively about some mind-numbingly banal topic such as ‘Books or TV’. Informational writing is not assessed at all, presumably because we cannot assume students know anything. How is the issue of writing style resolved? Well, according to Perelman, NAPLAN rewards students for using challenging words – whether they are the best words to use in a given context or not – it rewards them for using lots of adjectives to modify nouns and it rewards them for using connectives such as, ‘moreover’ and ‘on the other hand.’

These may once have served as useful proxies for expert performance, but now we all know what they are, they won’t any longer. This is a classic example of how rubrics fail.

The solution to this mess is twofold. Firstly, narrative writing should be assessed separately. Secondly, persuasive and informative writing should be set in topics selected from the previous twelve months of the Australian Curriculum. Although the Australian Curriculum is weak and content-lite, this would have at least two advantages. Firstly, it would provide an incentive to actually teach and assess the knowledge component of the curriculum, such as it is. Secondly, it would level the playing field. Children from privileged backgrounds would no longer be at a distinct advantage because their parents happened to buy them a book on / have a discussion about / visit a museum related to content relevant to the writing prompt.

Similarly, the reading assessment should also be based in the Australian Curriculum, similar to a proposal that is currently on the table in Louisiana.

Neither reading or writing are wholly generic sets of skills and we neglect this fact at our peril because it leads to invalid assessments.


9 thoughts on “How to improve Australia’s writing assessments

  1. David says:

    Greg, you are dead right about the Oz curriculum being content-light. My kids are working their way through high school (selective schools) and I can’t believe the impoverished content! I’ve mentioned in a comment on a previous post the reading work they get: happily it includes Shakespeare, but the first assessment asked for complex views on love and society: I wondered if this had been discussed in class; I also wonder if the curriculum includes examples of good literary and other non-fiction essays to give the students some work to inform their own practice and to aspire to. The Essay is a fine art-form to be cultiviated and therefore studied. I don’t think we studied any essays in my high school, but there are plenty of good ones that could be included.

  2. I use the primary curriculum a lot of the time for my work and you are completely right about the lightness! Although, I am not so sure lots of teachers would agree as I hear a lot of mumblings about the weight of the curriculum too. For my own children who are at local state schools, I go with the Seth Godin approach of all kids whether schooled or not should be homeschooled. Naplan is just another facet of a school system, be it public or private that is not enriching our children’s lives with nourishing learning experiences. There is so much good research and classrooms of well meaning teachers are still full of spelling lists made up of high frequency words, maths with no context and reading passages, because there is no time for the whole book.

  3. John Perry says:

    Perelman’s report is slowly making its way through the various avenues of public discourse (it’s a big document). However, I’m worried that Jennifer Buckingham’s take on NAPLAN is going to be taken too seriously. Her analysis (today) of NAPLAN’s deleterious effect on stakeholders carefully sidesteps evidence provided by research, even the research articles she cites. The latter point begs the question of how carefully she actually read it …

  4. John Perry says:

    That’s the one. And I will correct my erratum: “It raises the question.”

    The only one making unpleasant insinuations is Dr Buckingham herself, as she continues to wilfully ignore current research and gloss over important points from the research she does actually cite, like this shocker:

    Almost ninety percent of teachers reported students talking about feeling stressed prior to NAPLAN testing, and significant numbers also reported students being sick, crying or having sleepless nights.

    Which, apart from not even quoting, she dismisses with:

    Surveys of teachers and parents indicate that some students experience anxiousness but there is no evidence that this is any more than the normally heightened emotion associated with any new experience, or wanting to do well.

    Really? There could be evidence, if she compares studies that show the levels of stress associated with “new experiences” or “wanting to do well” (I’m tempted to accuse these phrases of being “weasel words” on par with “collateral damage”, but I’ll desist), with the levels of NAPLAN stress that are reported in Howell (2017), Rogers et al (2016), and Polesel, Rice and Dulfer (2014), amongst others. I’d like to know why she feels this comparison is unnecessary.

  5. Pingback: Tackling bias – Filling the pail

  6. Pingback: Testing is not the problem, it is part of the solution | Filling the pail

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.