Editorial |
Editing the JPH
Norman Vetter
There is a strong rumour, heavily denied by my family, that I wish to spend more time with them and will therefore be giving up editing our beloved journal shortly. It struck me that, in case I do not get another chance, I should discuss some of the curious byways of editing scientific journals.
 |
The editor's decision is final and possibly even right |
|---|
 TOP The editor's decision is... Does it help to...Can you train a...So is there any...If they take for...How to get published...Publish in a journal...References |
|---|
Many people, editors included, worry about whether they are really accepting the right articles and whether they have chosen the right reviewer for a paper. In the JPH a high proportion of the first sieving is done by the editors themselves.
The majority of the peer-reviewed clinical journals are edited by editors whose training prior to taking up post may be limited or nonexistent in editorial matters. Most are honorary, of course, so that the pressure of their day-to-day work may not make extensive training possible. One study found that ‘Clinical journals are usually edited by practicing clinicians who are self-taught and part-time editors, but willing to accept further training’.1
So, is the whole system unfair? A study of the subjective rating of reviewers by editors showed that these were moderately reliable and correlated with the reviewers ability to report manuscript flaws.2 However reviewer's rates of recommendation for acceptance were poorly correlated with editors' ratings of review quality or the reviewer's ability to detect flaws in a fictitious manuscript. Using 124 reviewers of a fictitious manuscript, the mean quality rating of the reviewer, as measured by the editor, was modestly correlated with the number of flaws they reported. Highly rated reviewers reported twice as many flaws as poorly rated reviewers.
However another, more recent paper suggested that the only significant predictors of quality were working in a university-operated hospital versus other teaching environment and relative youth; under ten years of experience after finishing training.3 All other variables in a study of 2,856 reviews of 1,484 separate manuscripts during a four-year study period showed no significance. They conclude; ‘Our study confirms that there are no easily identifiable types of formal training or experience that predict reviewer performance. It all sounds like a typically British system, but, interestingly, the USA has a very similar approach.
|
Does it help to stick pins in wax effigies of the reviewers? |
|---|
|
TOPThe editor's decision is...Does it help to...Can you train a...So is there any...If they take for...How to get published...Publish in a journal...References |
|---|
Authors in a study from the Annals of Emergency Medicine were not too struck with their reviewers, or the peer-review process in general.4 Of those authors asked, 576 authors (64%) returned the survey. Those whose papers were rejected without review were the least satisfied with the letter explaining the editorial decision. Among respondents whose manuscripts underwent full review, overall satisfaction was highly associated with acceptance of the manuscript for publication but not with quality rating of reviews. So authors want a result, no matter how erudite the criticism of their work.
The problem for authors is to finding out who the reviewers are for naming in their pin-sticking incantations, for authors are usually known to reviewers, but reviewers not known to authors.
A paper published in JAMA, as part of a series of papers written by BMJ staff, looked at 221 reviewers.5 These were randomly allocated to 5 groups. Groups 1 and 2 received manuscripts from which the authors’ names and affiliations had been removed, while groups 3 and 4 were aware of the authors' identities. Groups 1 and 3 were asked to sign their reports, while groups 2 and 4 were asked to return their reports unsigned. The fifth group was sent the paper in the usual manner, with authors' identities revealed and a request to comment anonymously. This study showed that neither blinding reviewers to the authors and origin of the paper nor requiring reviewers to sign their reports had any effect on the rate of detection of errors. The authors conclude that such measures are unlikely to improve the quality of peer review reports. It appears that pin-sticking does not work. We have all suspected for some years that reviewers are too thick-skinned for that sort of thing.
An interesting part of this study was that reviewers, blinded to the identity of the authors were asked to suggest who they might have been. I seem to remember that reviewers guessed correctly over half of the authors, or at least the centre to which they belonged. Such is the super-specialist world of researchers that there are few groups in any one field and all know each others work intimately.
|
Can you train a reviewer? |
|---|
|
TOPThe editor's decision is...Does it help to...Can you train a...So is there any...If they take for...How to get published...Publish in a journal...References |
|---|
The question of whether special training for reviewers improves the quality of reviewing was looked at by the BMJ some years ago. I was lucky enough to have been in one of the intervention groups.6 I remember the lunch, at BMA house, was extremely good. This made it harder to concentrate in the afternoon, with a possible effect on the results.
The study was a randomised controlled trial, where a third of the potential reviewers attended a training workshop while another third received a self-taught training package, focusing on what editors want from reviewers and how to critically appraise randomised controlled trials. The last third received no training. The quality of the reviews of three manuscripts sent to reviewers at four to six monthly intervals was evaluated using a validated review quality instrument; it identified the number of deliberate major errors identified, the time taken to review the manuscripts and the proportion recommending rejection of the manuscripts.
Both intervention groups identified significantly more major errors after training than did the control group but the evidence for benefit of training was no longer apparent six months after the interventions. They recommended a longer training period in future. An excellent idea: more good lunches.
|
So is there any such thing as a good reviewer? |
|---|
|
TOPThe editor's decision is...Does it help to...Can you train a...So is there any...If they take for...How to get published...Publish in a journal...References |
|---|
One rather terrifying paper has found that when a reviewer was less than 40 years old, from a top academic institution, well known to the editor choosing the reviewer, and blinded to the identity of the manuscript's authors, the probability that he or she would produce a good review was 87%, whereas a reviewer without any of these characteristics had a 7% probability of producing a good review.7 There was a negative but non-significant association between academic rank and review quality: 37% of full professors, 39% of associate professors, and 51% of assistant professors or fellows produced good reviews. This was a study from the USA, of course; it also seems like an excellent excuse for us old fogies to stop reviewing.
One approach to getting suitable reviewers, which is used by the JPH is to allow authors to suggest their own reviewers. Generally an author, presumably, then finds someone who owes him or her money and puts their name in the space provided. Whatever happens in practice a study shows that these reviewers do as good a job as reviewers chosen by the editor, in terms of the quality of the review.8 However those nominated by the author are more likely to recommend acceptance during the initial stages of peer review. I personally don't often use a reviewer suggested by the author, unless the potential reviewer is someone known to me.
|
If they take for ages is it good news or have they lost my paper? |
|---|
|
TOPThe editor's decision is...Does it help to...Can you train a...So is there any...If they take for...How to get published...Publish in a journal...References |
|---|
The use of Manuscript Central, a database and decision-tree online system, means that we editors can reject papers almost before they have been sent. Indeed I have had congratulations from an author on ‘the fastest rejection ever!’ There is a concern that a twitchy mouse finger can undermine the career of a potential Nobel Prize winner.
So hearing quickly is generally bad news. Taking for ages may be good news, but occasionally a paper does get lost. Well, they all look the same. Losing papers has become much less common since the advent of systems like Manuscript Central. The usual reason for delay is that a reviewer is taking a long time to respond or that no one wants to review it.
|
How to get published despite the reviewers and editor |
|---|
|
TOPThe editor's decision is...Does it help to...Can you train a...So is there any...If they take for...How to get published...Publish in a journal...References |
|---|
If you really want to get the work out there then try a letter to the editor. It is shorter and we editors rather like them, because they are short, can be controversial and entertain the readers who have a low concentration span.
A letter is most likely to be accepted if it responds to something in a previous issue of the Journal, for then others may be tempted to look up the previous issue and may even quote the paper, thus increasing the Journal's quotation index. However, there are often interesting small pieces of information which an editor is glad to publish, but which does not require the full panoply of an article or short report.
Another way is to become editor, when you get to write reviews on all the things that have made you cross over the years. Unfortunately they do take a bit of time to write (honest), and do not get you many Brownie points in the Research Assessment Exercise.
|
Publish in a journal or get to the punters? |
|---|
|
TOPThe editor's decision is...Does it help to...Can you train a...So is there any...If they take for...How to get published...Publish in a journal...References |
|---|
If you want people to know about something important, especially the general public, you will be better aiming at the more general press. Ph.com is excellent if you want to get to the public health specialists and trainees quickly. One study in the USA showed the value of sending health promotion material to suburban newspapers;9 it may not be worth much on the RAE, but the cuttings look better in your scrap book.
|
References |
|---|
|
TOPThe editor's decision is...Does it help to...Can you train a...So is there any...If they take for...How to get published...Publish in a journal...References |
|---|
- Garrow J, Butterfield M, Marshall J, Williamson A. The reported training and experience of editors in chief of specialist clinical medical journals. JAMA (1998) 280:286–87.
[Abstract/Free Full Text] - Callaham M. L, Baxt W. G, Waeckerle J. F, Wears R. L. Reliability of editors' subjective quality ratings of peer reviews of manuscripts. JAMA (1998) 280:229–31.
[Abstract/Free Full Text] - Callaham M. L, Tercier J. The relationship of previous training and experience of journal peer reviewers to subsequent review quality. PLoS. Med (2007) 4:e40.[CrossRef][Medline]
- Weber E. J, Katz P. P, Waeckerle J. F, Callaham M. L. Author perception of peer review: impact of review quality and acceptance on satisfaction. JAMA (2002) 287:2790–93.
[Abstract/Free Full Text] - Godlee F, Gale C. R, Martyn C. N. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA (1998) 280:237–40.
[Abstract/Free Full Text] - Schroter S, Black N, Evans S, et al. Effects of training on quality of peer review: randomised controlled trial. BMJ (2004) 328:673.
[Abstract/Free Full Text] - Evans A. T, McNutt R. A, Fletcher S. W, Fletcher R. H. The characteristics of peer reviewers who produce good-quality reviews. J Gen Intern Med (1993) 8:422–28.[Web of Science][Medline]
- Wager E, Parkin E. C, Tamber P. S. Are reviewers suggested by authors as good as those chosen by editors? Results of a rater-blinded, retrospective study. BMC. Med (2006) 4:13.[CrossRef][Medline]
- Maloney S. K, Petersen D. J. Placement of health promotion columns in suburban newspapers–an analysis. Public Health Rep (1985) 100:368–70.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||