Peer Review

Whenever researchers decide to write about peer review, it is customary to reference an anecdote about Einstein. Unlike most quotes attributed to Einstein, this anecdote is genuine and relates to a paper he submitted with a colleague, Nathan Rosen, to the editor of the journal, Physical Review. The editor then sent Einstein’s paper out to an expert for comment. When Einstein found out, he was annoyed that the editor had done such a thing without his permission. It is probably the only paper of Einstein’s that was ever sent for peer review, although, in this case, he perhaps benefited from it.

The purpose of this anecdote is to highlight that peer review is a relatively new idea. Pretty much all of the science we teach in school was developed in the absence of peer review and so to insist that it is a fundamental component of the scientific method is stretching things somewhat. And then there’s the fact that peer-review is also used for things that, well, I probably would not describe as ‘science’ such as the various unfalsifiable theories fashionable in the humanities. Real Peer Review is a Twitter account dedicated to highlighting the silliest and most amusing examples.

I have a number of experiences of peer review and two that stand out for illustrating its highs and lows. A paper I wrote with my PhD supervisors as a result of my PhD work was published in Educational Psychology Review, a well-regarded journal. As a result of the peer review process, we received feedback that ultimately made it a better paper.

In contrast, an opinion piece on metacognition I wrote for Impact, the journal of England’s Chartered College of Teaching, was rejected at peer review, at least partly on the basis of tone and the fact that the reviewers disagreed with some of my opinions. This then gave cover to people to ignore any arguments for or against my article and instead comment to the effect that all neophyte researchers experience disappointment in the peer review process and I would learn to cope. So, peer review can also act as a gatekeeper, guarding access to the sanctified lawns of the intellectual establishment.

As Real Peer Review demonstrates, just because something appears in a peer-reviewed journal, that doesn’t provide a guarantee of quality. Take, for example, the peer-reviewed journal Creative Education published by Scientific Research Publishing (SCIRP). It is the only education journal I can find that is published by Scientific Research Publishing, so if you have a paper you are thinking of submitting then you may be concerned whether your paper will find a home there. Just how creative does your education research need to be? This may be established by looking at the scope of the journal. It covers the following topics:

· Academic Advising and Counseling
· Art Education
· Blog Culture and Its Impact on Education
· Business Education
· Collaborative and Group Learning
· Curriculum Development
· Development of Learning Environment
· Early Childhood Education
· Education Administration
· Education Policy and Leadership
· Educational Psychology
· Educational Technology
· E-Learning and Knowledge Management
· Elementary Education
· Health Education
· Higher Education
· Innovative Pedagogical Models
· Language Education
· Learning Systems Platforms
· Media Education
· Music Education
· Other Areas of Education
· Quality Management of E-Learning
· Reading Skill Education
· Science Education
· Secondary Education
· Special Education
· Tasks and Problem-Solving Processes
· Teaching and Learning Technologies
· Web-Based Learning Platforms
· Youth Studies

So, I think you will be fine.

Creative Education has an unusual model where it publishes all its papers open-access and online. This is, in many ways, commendable. Obviously, Creative Education has to recover its costs in some way and so it asks for $999 in article processing charges. This is not particularly high compared to some publishers, but it is a cost that an author or their institution will need to meet. The Editorial board is multinational, with six members from the U.S., three from Australia and members from a range of others countries such as the UK, Brasil and Israel.

What sort of papers are published in Creative Education? Well, as you may imagine, they are somewhat eclectic. For instance, this paper appears to be a proposal for a research study, a type of paper I’ve not seen published before. On the other hand, this paper, albeit one I would strongly disagree with, seems more typical of education journals.

When you click on the ‘About SCIRP’ link and look for the contact details, there is no postal address listed. Instead, it says, ‘You can also contact us by e-mail, QQ or WeChat.’ QQ and WeChat are social media apps owned by the Chinese conglomerate Tencent.

The Wikipedia page for SCIRP provides a number of links. One is a story about the mass resignation of members of the editorial board of one of its journals. Another is to a list of Retraction Watch stories involving SCIRP, and another is to a 2012 paper by Jeffrey Beall that gives some background on SCIRP and comments on the quality of its operations. This is quite an old article and so perhaps SCIRP has changed since then. Nevertheless, if peer review simply means an article has been published in a journal such as Creative Education, what weight should we place on that?

As ever, the quest for a simple metric leads to complexity and confusion. The fact that a publication has been peer-reviewed means little and, as ever, we must read beyond the headline if we want to know what’s really going on.

Why have I returned to the subject of peer review? After writing my open letter to Ronald Sackville AO QC, Chair of the Royal Commission into Violence, Abuse, Neglect and Exploitation of People with Disability, a number of campaigners mocked me for referencing my own non-peer-reviewed article about differentiation. One suggested my article had been published in the ‘International Journal of Ballarat’ – a reference to this blog. So, that caused me to think again about the cultural power of peer review – the myth of peer review, if you like.

The reason I chose to reference that particular article on differentiation is because it is free to access and because there are not that many people making the case against differentiation. Mike Schmoker is someone with a similar view but I cannot imagine an article that was critical of differentiation being published in a regular education journal. So, I think we need to hear more from this side of the argument and, with that said, it would be an omission if I did not point out that I give differentiation a more extended treatment in my new book, The Power of Explicit Teaching and Direct Instruction, which is available to pre-order now from all good online bookstores.

Just like the International Journal of Ballarat, my new book is not peer-reviewed. But, you know, so what?


3 thoughts on “Peer Review

  1. David F says:

    Hi Greg–you might be interested in this study by the US Army in 2013 entitled “Comparison of Direct Instruction and Problem Centered Instruction for Army Institutional Training”

    Click to access a578566.pdf

    They mention differentiation in the context of “contrasting cases/invention” instructional design.

    • Thanks. That section is perhaps using ‘differentiation’ in a different way. I’d need to check the source. However, I am familiar with the contrasting cases/invention field – it’s actually relevant to my PhD. Sweller criticised some of the research on this in Constructivist Instruction: Success or Failure by saying the experiment vary more than one thing at a time.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.