Academic rigour and The Conversation

Here are some claims relating to education that have been made in recent weeks by Australian academics in The Conversation:

  1. Our declining results in international tests of literacy show us that our 15 year olds can decode but they can’t comprehend. [Source]
  2. Test results say nothing about teaching quality. [Source]
  3. A study that looked at how schools develop policies and practices to prevent behaviour problems found the following methods effective… [followed by a list of strategies]

The first two claims are obviously false. The only international reading test for 15 year olds is the Programme for International Student Assessment (PISA). This does not test decoding and so it is simply impossible to use it to infer that ’15 years olds can decode but they can’t comprehend’. Similarly, although there are many problems with using test results to infer teaching quality, it is clearly not true to claim that these results tell us ‘nothing’ about it.

The third claim about behaviour problems seems quite measured in comparison. Indeed, there is a link to the study in question. However, if you take a look at this link then you will see that it is a ‘framework’ that is extremely light on detail. Significantly, a strong causal claim has been made and in order to evaluate such a claim we need to be able to examine exactly what was done, how it was done and what was measured.

The detail is important for a number of reasons. Firstly, some of the points from the suggested list of ‘effective’ strategies look like they originate in progressive education ideology:

When students appear disengaged, staff work in collaboration with students to support the professional learning needs of teachers and develop engaging learning environments

Staff are committed and have support to enable them to follow up with students the day after if there has been an issue with behaviour. Teachers utilise behaviour management approaches that engage the student rather than punitive approaches that lead to alienation [my emphasis]

The origin of these ideas does not mean that they are necessarily wrong and that these strategies are necessarily ineffective. However, education has an unfortunate history of adopting approaches on ideological rather that empirical grounds. So it is critical to be able to evaluate the evidence supporting such claims.

And this issue is of real, international significance. Many supposedly ‘zero tolerance’ schools would be described by their detractors as using a ‘punitive’ approach and so these claims add to the political debate around such schools. Given that zero tolerance policies have been a feature of some Charter Schools and Free Schools, such claims also add to that debate.

I asked about the evidence in the comments section and received a very interesting response from the author, Anna Sullivan:

“Thanks Greg, that particular link goes to a framework which outlines the findings for this part of the BaSS research. The book that explains this framework is due to be published later this year by Routledge. This is the beauty of The Conversation- research can be disseminated much quicker that the usual peer reviewed publications.”

So the detail of the evidence is in a book that hasn’t yet been published.

The strapline from The Conversation is, “Academic rigour, journalistic flair”. This aspires to a higher standard than most news outlets. If writers are to take the claim to ‘academic rigour’ seriously then I don’t think they should make assertions that are different in substance to the ones that they would be prepared to make in a paper submitted to a peer-reviewed journal.

And no peer-reviewed journal would accept a claim based upon evidence that cannot be examined.

The Conversation is about injecting the voice of academia into the national and international debate. I worry that too many academics take an unintentionally patronising approach to the general public by assuming that the public will not understand the detail and expecting the public to take their assertions on trust. This may have been appropriate for government information films of the 1940s but I think we can do a lot better now.

Advertisements

9 Comments on “Academic rigour and The Conversation”

  1. Mike says:

    I’m a relative newcomer to that website, but my abiding impression is that the vast majority of those writing for it or commenting on it (or both) are patronising in the extreme and stolidly intellectually conformist.

    • Alex Brown says:

      I have an honours degree in a social ‘science’ and I no longer think qualitative research is relevant. I could be reconvinced of its place, but for now it’s just rhetoric and ideology.

      Ive got three degrees and im starting to think University is a bad investment these days. Extraordinary, I know.

    • Stan says:

      Thanks Mike I do like to patronize blogs that conform to a stolidly intellectual idiom.

    • Stan says:

      Still thanks Mike. When the ideology is to engage in informed discussion and not be afraid to challenge the status quo with evidence based arguments you’re still handing out compliments.

      If you are attempting a criticism then it is going to come across as an own goal if you can’t be more specific about there being something else to this ideology of which you write and tell us what is wrong with it.

      You’re coming across a bit like one of those critics of the consensus among climate scientists. Those scientists and their dang scientific ideology of going where the evidence takes them and ignoring the short term economic pain their conclusions might lead to.

      • Mike says:

        You’re not making a great deal of sense there. What you’re talking about in your second sentence is an attitude, not an ideology. As for evidence-based arguments, Greg has just gone to some trouble in this post to explain how some of the claims on that site (I’m just referring to the education part of it, by the way) are NOT evidence-based. And these tend to be the claims that are cheered to the rooftops by the majority of the commenters, chiefly because they reflect the prevailing progressivist ideology that is standard-issue in university education departments and has been for decades. (Which, by the way, makes your observation about “challenging the status quo” rather puzzling.)

        And re your last para, for now I might just mention that the scientific method is not an ideology either.

      • Stan says:

        Hi Mike,
        I think I got the wrong end of the stick. I thought you were talking about this site not that site.

        But an ideology doesn’t have to be a negative. You can have worthwhile, worthless and expensive ideologies. While the scientific method is not an ideology. Adhering to it would be a worthwhile ideology.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s