Accuracy, opinions and tone

I wrote an article for Impact, the magazine of England’s new Chartered College of Teaching. They declined to print it. They had sent it out to two sets of reviewers who took issue with a number of aspects of what I had written. Given that it was essentially an opinion piece, I expected comments on accuracy: Were my figures correct? Had I correctly explained statistical significance? And so on.

I was surprised to read comments objecting to my tone or simply disagreeing with my opinions. Tone is largely subjective. Anyone reading my piece who is personally invested in metacognition and self-regulation will find the tone confronting. Others will simply find it playful.

When I commented on Twitter about the fact that the reviews took issue with some of my opinions, the College Twitter account said that I was wrong:

This is what forced my hand and made me decide to release the anonymous reviews.

Following this, some defenders of the College engaged in a Pythonesque Twitter thread where they attempted to deny the obvious. As far as I understand it, they claim that the reviews did not take issue with my opinions because they gave good reasons for taking issue with my opinions. Or something. For me, this peaked with the following exchange:

However, what has perhaps been lost in this process is the notion of accuracy. When I was sent the final set of reviews, I was given a couple of days to revise my article for tone and accuracy. I declined, partly due to the fact that I didn’t see the need and partly because of the time I was given.

One reason I didn’t see the need is that there were so few comments relating to accuracy. Indeed, one reviewer seemed to indicate that s/he lacked the knowledge to comment on accuracy and instead suggested my tone was ‘Clarksonesque’. Those who did made comments about accuracy appeared to be wrong.

For instance, one reviewer claimed, “the assertion that the Toolkit indicates that a teacher will get 8 months’ progress is wrong and misleading.” On the 13th April, the Toolkit revised this figure down from 8 months to 7 months, but at the time of writing, and at the time the review was written, it was indeed 8 months. So what was wrong with my claim? It is obviously a nonsense that anyone could expect to get 8 months progress from implementing one of these diverse approaches but this is what the Toolkit suggests and this is one of the reasons I am critical of it.

One reviewer noted that my opinion contradicted that of the Education Endowment Foundation (EEF) but didn’t really resolve the issue, despite seemingly showing sympathy for the EEF’s position. The comment seemed more of a caution to the editor than something I could respond to.

There is one important claim about accuracy in the reviews that could be a valid point. After criticising my ‘selective’ use of a Dylan Wiliam quote, a usage that Dylan Wiliam did not object to in his own review, Reviewer four mentions, “…references to statistical significance which are of little relevance in cases where effect sizes are reported with confidence intervals.”

Jim Thornton, professor of obstetrics and gynaecology at Nottingham university and an expert in randomised controlled trials, took up this point in a comment on an earlier post:

“There were no confidence intervals around the effect sizes in the P4C trial, nor in the EEF meta-cognition and self regulation review. In the technical appendix the range of the different effect sizes is reported, which might have confused reviewer 4, but this is quite different from the confidence interval around the effect size. Reviewer 4 is correct that confidence intervals would have negated the need for tests of statistical significance. The problem is they weren’t reported. The very point that Greg was making.”

So I’m not really sure how I could have revised the article for accuracy.


Notes: You can find my original submission, prior to any amendments I made on the basis of the first three reviews, here. You can find my response to the first three reviews here. The sixth response should read ‘The fact that teaching methods..” but I’ve left it as submitted for transparency.

Advertisements

22 thoughts on “Accuracy, opinions and tone

  1. I think Impact is right that the tone of your article was too hyperbolic. You should have modified the tone, while continuing to argue your case on substantive points. You threw your toys out of the pram when there was no reason, apparently because you couldn’t be bothered (or were too proud) to submit another revision.

      1. But tone is not subjective. If you suggested that the tone of a Sun newspaper editorial was indistinguishable from the tone of an academic paper, I would disagree and (with time and inclination) would evidence my position by reference to vocabulary, use of adjectives, rhetorical devices etc. If you said that a preference for one tone over another was a matter of personal preference, again I would say you were wrong on the objective grounds that formal registers are demonstrably more useful to a forum that attempts to focus on a discussion of evidence, and not on achieving an emotional impact.

        As for the passages which are inappropriate, I would start with the use of sarcasm in the first paragraph: “The evidence for its effectiveness is even stronger than the evidence supporting the use of feedback. It’s a no-brainer then. Off you go and do it!” Most of the rest of the article is written in similar vein. The claim that “the EEF have created a monster” qualifies for hyperbole in the strict sense.

      2. You’ve just shown how subjective this all is. I wouldn’t call the first paragraph sarcastic, I would call it engaging and playful.

        And as for the idea that “The EEF have created a monster” is hyperbolic, this is in direct reference to the analogy of the chimera that I have used throughout the piece and that was present in the submitted abstract upon which the decision to ask me to write the full article was made:

        https://gregashman.wordpress.com/2018/01/26/my-submission-to-the-chartered-colleges-impact-magazine/

        Reviewer 1, who did not like my article much, thought I should make *more* use of this analogy.

        The Chartered College have every right to print whatever they like but I do think that shedding light on this process has been a valuable exercise.

      3. Greg,

        I have no problem with you saying that the tone is playful or that it works brilliantly well. But if you say that it is not sarcastic, then I think that you are objectively wrong. If you say “it’s a no brainer” and mean “its the opposite of a no-brainer”, then that is a clear example of a literary device called sarcasm.

        I don’t know what tone the editor of Impact wants in the new publication – and maybe while they are in a state in which there are no precedents, the reviewers are not sure either; maybe even the editor is not sure.

        Nor would I be surprised if different reviewers contradict one another. There is no expectation that reviewers will agree with each other and the job of the author is either to modify the article or to rebut any and all criticisms. The process of justifying your piece in the face of criticism is surely how peer review is supposed to work. And it seems to me to be entirely legitimate for the reviewers to say that the tone was not appropriate for a formal journal.

        As I said in my first comment, I think the basic point that you were making is a good one and needs to be made. I just don’t think you were right to criticize the journal of impropriety when there really wasn’t good evidence for it and it meant that you lost the chance to force your critics to reply to what seem to be your good arguments against the charge of inaccuracy.

        Crispin.

      4. Because irony is generally more indirect or situational. While the definition of sarcasm (https://en.oxforddictionaries.com/definition/sarcasm) fits your usage very precisely.

        But it is not significant. If you don’t like my tone, I will be very happy to compromise on “irony” in order to facilitate the conversation (as you should have done). My substantive point (which is what matters) still stands: irony is not appropriate to a formal, peer-reviewed journal.

      5. Sarcasm signifies contempt according to most definitions:

        https://en.oxforddictionaries.com/definition/sarcasm

        You have projected contempt onto what I wrote, presumably due to the subjective way that you read my piece. Irony is a much more objective term for the device used. Which continues to prove the subjectivity of tone. As I have maintained all along, someone with no axe to grind would find the tone playful. This is why arguments about tone are mostly fruitless.

      6. As I say, this argument is not significant. My personal judgement is that the tone is not appropriate to a formal journal (regardless of whether you call it sarcastic or merely ironic) – but my judgement is not significant in this case either. What *is* significant is that the editor, advised by reviewers, has every right to insist on what he judges to be appropriate tone in his journal. The question of tone is not essential to your message, so there was no good reason for you not to modify tone. And, so long as you had not addressed the editor’s reasonable requests on tone, and not fully rebutted any accusations of inaccuracy, you had no justification for making what is a very serious, public accusation of impropriety against the editor.

      7. My understanding is that you are claiming that Impact is not publishing you because it (or its reviewers) don’t like your views. I would view that as improper behaviour in a peer-reviewed journal, likely to damage its reputation. Am I wrong?

      8. You are making a few too many leaps there. Accusing the editor of Impact of impropriety is a grave accusation. I have not made such an accusation and you should withdraw your claim that I have unless you can back it up with more than your own speculative inferences.

      9. You have made a very serious claim. You hav claimed that I have accused the editor of Impact of impropriety. I have not. You have then attempted to substantiate this claim by stating that I have claimed that, “Impact is not publishing you because it (or its reviewers) don’t like your views.” I have not claimed this. Even if I had claimed this, I disagree that this would be equivalent to accusing the editor of Impact of impropriety because magazines have a right to publish what they like. Given the seriousness of your accusations, and the fact that you have not withdrawn them when asked, your comments on this blog site will no longer pass moderation until such a point as you withdraw these accusations.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.