An author, Kjell Gundro Brurberg appealed the rejection of his manuscript. He was offered an opportunity to nominate additional reviewers, but to ensure they did not have conflicts of interest. What happened next…
Kjell Gundro Brurberg happens to be an author on a controversial Cochrane review about which there are serious concerns about conflict of interest and coordinated outcome switching across trials.
He is also an author on a forthcoming Cochrane review for which Cochrane refuses to share data for independent reanalysis. This whole incident adds the importance of Cochrane releasing the data from that review for independent reanalysis, which the organization is currently refusing to do.
Now Brurberg is misrepresenting what happened in a blog post at Mental Elf, a blog site that has consistently shown itself to not vet what is posted by bloggers.
I will document in this issue of Quick Thoughts what really transpired. But his blog post indicates that Brunberg just doesn’t get how he blew an opportunity to publish his manuscript with his bad behavior.
Giving authors the opportunity to nominate reviewers has benefits to both the authors and to editors, but it depends on trust and the option can be abused by authors.
The whole story is part of a larger narrative of how hard it is to obtain an independent critique of the PACE trial of cognitive behaviour therapy and graded exercise for chronic fatigue syndrome.
I know from experience as a Senior Consulting Editor at the Journal of Health Psychology. Under Editor-in-Chief David Marks, the journal accepted an editorial commentary critical of the PACE trial and invited responses from a variety of perspectives. The journal has endured repeated assaults on its editorial independence and integrity since.
Here is my updated account of the PACE investigators’ pressures on Journal of Health Psychology to retract portions of the published commentary and to issue a correction acknowledging the author had an undeclared conflict of interest. Ah, the kettle calling the pot black.
But before we delve into the details of the current incident, let’s discuss the practice of allowing authors to nominate reviewers.
Background: Authors being allowed to nominate reviewers
Authors can be instructed not to nominate reviewers with obvious conflicts of interest or indications of being unlikely to provide an unbiased review. Even, better editors can ask authors to indicate explicitly that the reviewers they have nominated are free of conflict of interest. Authors declaring there is no conflict when there is an obvious conflict of interest can be seen as tantamount to scientific misconduct.
Until recently, authors were routinely asked to nominate reviewers for their manuscripts. What had been a common practice became controversial and was outright stopped at many journals when a major scandal in peer review occurred in 2013- 2015.
In what turned out to be only the first wave of continually uncovered problems with a number of journals, Springer and BMC (which Springer now owns) undertook an investigation and retracted 50+ papers for peer review manipulation and other issues. Commenting to Retraction Watch, one publisher said:
Alongside investigation into the identified papers, we have taken action to ensure that no further compromised papers can continue through to publication. BioMed Central has changed our policy on suggesting peer reviewers so that authors may do so in a cover letter with evidence of the peer reviewers’ authenticity.
Once the crisis went public, at least one journal continued to ask authors to recommend reviewers, but then made sure that that papers didn’t go out to any reviewers whom the authors had nominated. Ouch!
The crisis had seemed to ease, but a new record was recently set with a major publisher retracting more than 100 studies from a cancer journal over fake peer reviews.
Not surprisingly, there is evidence that reviewers who are recommended by authors are more likely to give positive reviews.
Our results agree with those from other studies that editor-suggested reviewers rated manuscripts between 30% and 42% less favorably than author-suggested reviewers. Against this backdrop journal editors should consider either doing without the use of author-suggested reviewers or, if they are used, bringing in more than one editor-suggested reviewer for the review process (so that the review by author-suggested reviewers can be put in perspective).
Obviously, editors need to be diligent and skeptically probe author-nominated reviewers. But relying on some of these nominations can be helpful if an editor, particularly if nominations are screened and provide only a portion of the reviewers for a manuscript. Flagging that a particular reviewer was nominated by an author is also helpful in the editor’s interpreting discrepancies in a set of reviewer recommendations for publication.
Presumably reviewers recommended by authors will be familiar enough with the topic of a manuscript to accept a request to review and provide in a timely fashion.
Authors may actually be best qualified to identify reviewers for their particular areas of research, even if editors have to evaluate the potential for bias and conflict of interest.
Though I’m most comfortable handling manuscripts in my areas of research, I am serving as academic editor for a megajournal with a broad interdisciplinary focus. I can’t expect to be on top of every nook and cranny of every subfield.
I also may not be as familiar as authors are with the up and coming researchers in a particular area of research who do not yet have a large number of publications, but who have an impressive brand new or “early view” paper or two out.
Relying on some reviewers nominated by authors can also help the balance the Anglocentric –and in some fields –older male bias of editorial boards.
Sure, there are problems with relying too heavily on reviewers that authors suggest, but let’s not get too misty eyed nostalgic about peer review in the old days when editors tended to rely just on people that they only personally knew.
There was a lot of old boy cliques looking after each other that could be the formidable obstacles if you wanted to challenge dominant theories. Some specialized journals were dominated by senior investigators who kept out threats to their theories or particular findings.
That is certainly still a problem in particular areas of research, but with more democratization of peer review and transparency, it gets a bit easier to uncover and challenge old boy cliques.
But unless publishers provide user-friendly tools to evaluate suggestions or can hire private investigators, editors have to rely to considerable extent on simple searches of published papers and trusting the report of authors to evaluate whether reviewers are appropriate and that they have no conflict of interest. Instructing authors ahead of time that nominations have to be free of conflict of interest is important.
I once was suspicious of an author nomination in an appeal of a rejected manuscript in a highly contested area of research. Google Scholar revealed no shared publications, but a broader internet search quickly yielded wedding pictures with the reviewer as the author’s best man.
This brings us to a recent problem we faced at the Journal of Health Psychology. The Senior Editor, David Marks, rejected a manuscript based on a rather thorough negative review. The author appealed this decision and was offered the opportunity to nominate additional reviews. However, because the manuscript concerned the contentious issue of the PACE trial, the author was given the responsibility to pre-screen his nominations for reviewers and indicate in writing that there were no relevant conflicts of interest.
The letter to the author rejecting his nominations because he misrepresented the conflicts of interest of reviewers
Dear Dr. Brurberg:
I write you in regards to manuscript # JHP-17-0254 entitled “A PACE-gate or an editorial without perspectives” which you submitted to the Journal of Health Psychology within an appeal procedure. Your manuscript is rejected due to your misrepresentation of conflicts of interests.
I have sought advice from my Associate Editors and this email is therefore copied to two of them.
I was going to wait until the end of the month before letting you have a decision, but new information came to light about Dr. Y that makes further delay unnecessary. The three reviewers that you recommended were supposed to be neutral, independent experts with no known conflicts of interest. Unfortunately, however, one declined the invitation to review (X) and the other two (Y and Z) have objectively proven conflicts of interest.
In light of your appeal, you were given the generous opportunity to have your manuscript re-reviewed by one or more impartial experts chosen by you. It is highly disappointing and curiously naive that you have attempted to subvert the appeal process by recommending people who are strongly conflicted, in one case (Dr. X) by his own admission is an associate of the PACE investigators. You stated that Prof X is: “Interested in medically unexplained diseases. Holds the needed distance to the ongoing PACE debate.” The latter could not be further from the truth. In his email on 26 April 2017 Prof X stated:
Bit of a curve ball this one I suspect!
Having been trolled briefly by [ the author on whose paper a comment was submitted] a while ago I might have a personal axe to grind, and having been supervised by Michael Sharpe (who may or may not have anything to do with this manuscript) between 5 and 10 years ago I would probably be regarded as irrevocably conflicted by the anti-PACE-ists. Mind you, I’ve also sat next to Peter White at a couple of (enjoyable) conference dinners – even that probably renders me tainted to some eyes.
Anyway, I’m perfectly prepared to be grown-up, reflectively self-aware, and as neutral as possible, in carrying out a review for you. But if you do open peer review you will probably be trolled for asking me.
If you still wish me to review, then please just let me know. But I thought I would give you a more than usually full COI summary first!
Hardly, a description of a reviewer who is likely to be independent and unconflicted.
In the second case, Dr. Y, has a clear COI by association with pro-PACE researchers through joint work and publication. You stated that Y has: “Expertise in exercise therapy and CBT. No COI with regard to commentary authors.” Yet I discovered on Google Scholar that Y was actually a visiting staff member of the School of Psychology at the University of Southampton. The working visit was made possible by a grant from the Dutch MS Research Foundation (Stichting MS Research). The original RCT was funded by a grant awarded to R. Moss-Morris by the New Zealand Neurological Foundation.” I am sure you don’t need me to tell you that Prof Moss-Morris is closely connected to the so-called ‘Biopsychosocial Model’/’Dysfunctional Beliefs Model’ of ME/CFS advocated by the PACE Trial team. This fact is evidenced by:
[A specific publication co-authored by someone central to the PACE controversy removed because it identifies the proposed reviewer.]
It is important to consider, in addition, your own conflicts of interest as a person who worked for the Cochrane Collaboration in the analysis of individual data on exercise therapy for CFS including data from the PACE Trial and from studies by Moss-Morris (2005) (already mentioned above). The relevant reference is:
Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database of Systematic Reviews 2015, Issue 2. Art. No.: CD003200. DOI: 10.1002/14651858.CD003200.pub3.
It is not often in my experience that an author misrepresents the facts about his/her recommended reviewers in such an audacious and palpable manner. If you lie these days, exposure is only a few clicks away. Norway is a small country, a country that I dearly love, not a place I normally associate with cheats and rogues. That impression just took a nose-dive. You have wasted a lot of my time and you won’t be given a 3rd or 4th chance. You have already blown it.
I recommend that you reflect on the ethics and professionalism of your actions and the potentially serious consequences for your professional career. Better luck next time! But please don’t try it on again with this journal.
David F Marks PhD
Journal of Health Psychology
The authors’ response
On 26 June 2017 at 09:02, Brurberg, Kjetil Gundro <KjetilGundro.Brurberg@fhi.no> wrote:
You are right, this is completely my fault. It is obvious that I should have known that your ‘journal’ only accept reviewers who disregard the PACE-trial as well as the PACE-researchers. It would have helped me, though, if you had stated this policy when you first invited me to write this comment.
Kjetil G. Brurberg
Some past posts relevant to complaints about the bias of the Cochrane review and my efforts to obtain the data from it for independent re-analysis. Unfortunately, despite advocating that others share data for independent reanalysis, Cochrane refuses to share.