Why I don’t know how PLOS will respond to authors’ refusal to release data

The simple answer is that I am not involved in the decision-making process and it is a bit opaque to me.

The more elaborate answer:

  • plos oneThe strategies I used to obtain the PACE data from the PLOS paper are only what are available to anyone else.
  • Based on what I know, I do not expect to appeal King’s College, London’s decision through the Information Commissioner’s Office  because my original request was not made under the Freedom of Information Act. So, I particularly depend on PLOS’ response.
  • I am one of thousands of PLOS One Academic Editors, as well as a blogger at the PLOS blog Mind the Brain. Yet, this does not give me any special standing, any special influence, or even any special understanding of the process by which PLOS decides how to respond to the refusal to release the data.
  • My reporting of all that is transpiring serves to highlight the importance of data sharing and policies to make sure that authors do indeed share data when requested.
  • I am also modeling how to proceed, at least with the PLOS journals when authors do not share data. The PLOS family of journals is years ahead of most other publishers in terms of their explicitness and transparency. I invite others to try what I’m doing with other journals and report the results.
  • We may have to settle for these accomplishments, if the data are not forthcoming, but I am more optimistic than that.

On November 13, 2015, I made a formal request to Paul McCrone, the first author of a PLOS article  to provide the data needed for re-analyses and sensitivity analyses  to see if results held up with different assumptions. I got a quick reply from that my request was being considered under the Freedom of Information Act.

I cc’ed PLOS One in my request. I promptly received an acknowledgment from a staff member. My request was assigned a case number.

On December 11 2015, I received a letter indicating that King’s College declined my request for the data.

I had previously called for release of data from the PACE paper published in The Lancet.  I also have called for statements concerning the availability of clinical trial data to be incorporated into the consent forms soliciting patient involvement in research , as an extension of a BMJ patient-oriented initiative.

But the request to PLOS was different than other request for the PACE data because it involved a journal that required a commitment to data sharing as a condition for publication.

King's college London

King’s College, London

The reasons given for King’s College’s refusal to release the data makes fascinating reading.

The King’s College letter deserves a read. Personally, I think they brought disaster down upon themselves by rejecting my request with the excuse that:

 

 

There is a lack of value or serious purpose to your request. The university also considers that there is improper motive behind the request.

The letter goes on to identify the motive and purpose behind this request as polemical and then described the request as “vexatious”.

Since then there has been lots of outraged expressed from a broadening group of academic and citizen scientists, with vigorous debate of the specifics of the letter. It’s been quite gratifying that calls for the release of data are coming from outside the UK and outside the patient community. There is no doubt that the international scientific community feels there are some key issues for them at stake.

The powerful Retraction Watch highlighted my blog in its Weekend Reads:

King’s College London doesn’t want to release data to James Coyne from a study of chronic fatigue syndrome. See if the absurd reasons make your blood boil as much as ours: “The university considers that there is a lack of value or serious purpose to your request. The university also considers that there is improper motive behind the request. The university considers that this request has caused and could further cause harassment and distress to staff.” And here’s more from someone else who submitted a related request data.

And that was followed by tweets by a number of champions of open science and data sharing.

 

brian nosek

JP de

if hiding data bothers you

lakensAlready, the chain of events has drawn tremendous attention to the value of data sharing policies, but the need to have means of enforcement. Previous to the PACE investigators’ stubborn refusal to share their data, apparently a lot of scientists felt that the data sharing problem had been solved by requirements of governmental and foundation funding bodies.

When I’m asked how PLOS will respond

I am frequently asked how PLOS journals will respond. I have the benefit of having read their exquisitely documented policies requiring the sharing of data for all articles published in PLOS One, PLOS Medicine, and other fine open access journals. I can reply to queries with what I have read.

But honestly, I don’t know how PLOS will respond because I am not involved in the decision-making process. Like anyone else, I can go on the Internet to discover the rules of the game for asking for data and for how to complain when when the data are not received. I thus have an advantage over those who didn’t know there were such options or how to exercise them. But I’m in uncharted waters.

But right now, like everyone else, I’m waiting to see how PLOS chooses to respond. I don’t know of another precedent in which authors refuse to provide data for a PLOS article, so I wouldn’t be surprised if a policy remains to be formulated. I have no doubt, however, that the issue will be carefully considered by PLOS and the principles behind a decision-making will be well articulated in a future policy statement. We need to be patient.

Disclosure

I happen to be a PLOS One Academic Editor, but there are thousands of us, certainly over 4000. You have to appreciate the hugeness of PLOS: last year PLOS one published 30,000 papers or 85 papers a day, making it the world’s largest journal, in terms of output.

I visit universities around the world giving talks. I inevitably try to recruit new PLOS One academic editors, particularly for subject areas where there has not previously been established expertise and yet where I want papers to be published after informative peer review. I encourage faculty, when they are asked to do a review for PLOS for which they do not have time, to supervise an advanced PhD student or postdoc doing a review as an educational experience.

A year ago I was in Hong Kong giving talks and providing evaluations of PhD student presentations, with the prices being all expense paid trips to international conferences. I was surprised to discover that one of the other judges was a senior staff member of PLOS based in London. That is the only time that I have met a senior editor. I know the names of only a few of them and certainly could not pick them out of a police lineup.

PLOS has regular events at conferences where anyone can meet a senior editor. But have never had the pleasure of being at one of those conferences at the same time as a PLOS representative.

Pascali's_Island_film_posterMuch of the time my editing of PLOS feels like the Ben Kingsley character in the 1988 movie Pascali’s Island. Kingsley files detailed reports with unfailing regularity about what he observes at Nisi, a small Greek island under Ottoman rule, but does not get a response. At least I am confident that someone at PLOS reads my editorial recommendations (I only recommend, I do not decide), but there is a remoteness to the whole process when I have never seen the people on the other end.

I am optimistic about the crisis of the King’s College refusing to release data ultimately being resolved. The release of the data will allow not only me, but others to provide independent evaluations of how reasonable the authors’ interpretations made in their article are.

What has already been accomplished

I judge my efforts as already being tremendously successful in highlighting the issues of data sharing policies and finding ways to ensure there is not such a hassle when the next request is made.

Many people are now reading the PLOS statements about data sharing, although I suspect that many people voicing strong opinions in the discussion have not read these sources. They should read them and demand that the policies be adopted elsewhere.

Prior to requesting the data from the PLOS article, I’ve been involved in a number of reanalyses of published data. One set of re-analyses  revealed that a loving-kindness meditation intervention did not have a significant effect on physical health outcomes, despite already being heavily marketed in workshops and corporate presentations. The original article had not even acknowledge that it was a randomized trial until the supplementary materials. Another set of reanalyses showed that a complicated set of 53 multiple regression equations claim to demonstrate that distinct dimensions of psychological well-being are differentially correlated with levels of expression of a selection of genes, which in turn, are  associated with distinct forms of immune response produced equivalent results when random numbers were substituted for the data from the study.

Personally, I believe that such reanalyses are just as important, perhaps more so than single-study declarations of breakthroughs that often prove premature or simply wrong. I am committed to promoting such re-analyses as a suitable contribution for junior academics building their careers.

An Expression of Concern and a retraction as possible penalties if the data are not forthcoming

I was particularly delighted when Groningen biologist Klaas van Dijk– who is not actually at my University and whom I have never met – provided an excellent letter calling for an Expression of Concern from PLOS that I have not received my data . In the letter he calls for the retraction of the article if the situation is not rectified within a month. It’s a model response, which I’m sure will have opportunities to use at other journals in the future.

I was also delighted at the very intelligent comments left at the PLOS article and even that the opportunity to make comments as been highlighted. I appreciate in particular a well articulated comment left by Alem Matthees who is the publicity-shunning source of an earlier request for PACE data from a Lancet article. The request was rejected by Queen Mary University of London. Matthees appealed to the UK information Commissioner’s Office. The Commissioner then ordered release of the data, but that decision was also appealed by Queen Mary University of London.

The widely acknowledged untrustworthiness of the scientific literature is harder to rectify if such a ridiculous bureaucratic process in order to access data to reanalyzed.

And then there are the comments at PubMed Commons. Anyone can leave comments there if they can claim to have been an author on any one of the 25 million articles listed in Pub Med, and register with either their email in that article or their Google account. And if you have never published a paper listed in Pubmed, you can still comment through a recognized PubMed Commons journal club.

I posted the letter by which I requested the data in a blog and I hope it will be used as a model by others. However, the basic structure of the letter did not originate with me, I borrowed a model that my graduate student at Groningen, Nick Brown had used in a request that did not involve me at a journal other than PLOS.

Maybe eventually I will assemble a toolbox for obtaining data and appealing when the data are not forthcoming . Certainly the letters by Klaas and Nick will be part of that an appropriate credit will be given.

My blogging

Besides being an PLOS One Academic Editor, I am a regular blogger at the PLOS blog, Mind the Brain. You can read my self-description at the blog site.

But I should make perfectly clear that any opinions I express at PLOS Mind the Brain are entirely my own and should not be construed as the official policy of PLOS or even anything with which the management of PLOS would agree.

For instance, I recently ended a blog post with the statement

The views expressed in this post represent solely those of its author, and not necessarily those of PLOS or PLOS Medicine.

And I previously provided at my secondary blog, Quick Thoughts much the same message that I am conveying in this blog post :

Note: Although I am one of thousands of academic editors for PLOS One, I have sole responsibility for my blog posts, which cannot be construed as reflecting the opinions of any institutions with which I am affiliated, including PLOS. In requesting the data for an article that appeared in PLOS, I am exercising the same rights to anyone in the world has with respect to this open access journal committed to transparency and data sharing.

I will keep repeating this message to avoid confusion. I characteristically express strong, irreverent opinions, but they are strictly my own.

A troublemaker certified by the Cochrane collaboration

My colleagues and I questioned how the Cochrane collaboration incorporated investigator conflict of interest as a formal risk of bias in assessment of papers for inclusion in systematic reviews and meta-analyses. We recommended better recognition be given to what could be a source of exaggerated claims . The Cochrane collaboration responded by giving us the Bill Silverman award which is named after a recognized troublemaker in the early days of their collaboration.

I suppose taking up authors on their commitment to sharing their data can be seen as just another form of troublemaking. I’m sure my request is received with at least some ambivalence by the administrative staff of PLOS, with the likely necessity of formulating and disseminating yet another policy. If so, it’s not the first time I have challenged the family of journals with the result a change in policy. I see myself as an ethical hacker  of sorts: exposing faults in a system so that new solutions can be developed. But I really want to reanalyze the PACE PLOS data.

Advertisements

11 thoughts on “Why I don’t know how PLOS will respond to authors’ refusal to release data

  1. Miguel Roig ( http://facpub.stjohns.edu/~roigm/ ) wrote on December 12, 2015 at 5:34 pm at http://retractionwatch.com/2015/12/12/weekend-reads-what-do-phds-earn-does-peer-review-work-collaborations-dark-side/#comment-870288 :
    .
    “I don’t know about a retraction at this time, but perhaps PLOS editors are considering issuing an expression of concern. They have done so in at least one occasion in the recent past for an instance in which authors failed to share a strain of bacteria. The PLOS ONE Editors (2014) Expression of Concern: Bacillus pumilus Reveals a Remarkably High Resistance to Hydrogen Peroxide Provoked Oxidative Stress. PLoS ONE 9(7): 3100716. doi:10.1371/journal.pone.0100716 “.

    Like

  2. Thank you for taking the time to explain in clear layman language for those of us with Myalgic Encephalomyelitis. Our cognitive issues & brain fog makes reading very difficult. There is no accountability when it come to treating ME in the UK & Europe. Denmark believes that ME = Insanity. This is not a disease that you can pretend to have. Until now we have been banished to ME Purgatory wishing someone would help us. Thank you Professor James C Coyne for helping us. We appreciate the time you take to explain what you are doing and why. It helps us to study your suggestions so we can understand how to help ourselves. Thank you for bringing this to the attention of other research scientists – I’m enjoying their comments. Out of 20 million PWME there are only a couple thousand advocates who do this from a deep passion. Because we are all sick we only add up to a few hundred advocates advocating at the same time. We are simply too sick to be harassing or sending death threats. A fellow Canadian with severe ME for over 30 yrs told me yesterday that this is the best Christmas she has ever had because of this strategy playing out right now. That’s a huge thank you to all of you.

    Like

  3. How to re-analyze the PACE trial in 4 easy steps

    (Note: Pre-analysis preparation- place large pillow on floor beside work area)

    Step 1: Re-analyze PACE trial data strictly according to trial protocol then compare results with what has been reported by authors thus far. Bonus points for analyzing percentage of prior results that included changes from trial protocol/statistical analysis plan, whether changes from protocol/analysis plan made it easier or harder for authors to report improvements, positive results, etc.
    Step 2: See if participants potentially qualifying for protocol-specified ‘positive outcome’ and/or ‘recovery’ criteria show associated improvements in any of the objective measures of outcome included in the trial, eg. 6 minute walk test, self-paced step test, hours worked, employment losses, overall service costs, welfare benefits or other financial payments, etc.
    Step 3: Include discussion if patient self-report results possibly affected by placebo/nocebo response since not only was trial unblinded but CBT/GET groups actually told as part of ‘treatment’ that recovery is possible and CBT/GET will help facilitate recovery, etc, while APT group told basically not much you can do.(1,2) Examine whether similar practice would be acceptable in trials on homeopathy, acupuncture or any other intervention.
    Step 4: Fall onto pre-arranged pillow and lolololololololololololololol!

    1. “If learning to cope with CFS is the jointly agreed maximal goal of treatment, patients will engage with treatment accordingly. If the therapist suggests that recovery is possible, the patient expectations are raised, which in turn may lead to a change in the perception of symptoms as well as disability. This is also the essence of the placebo response.”
    Knoop H, Bleijenberg G, Gielissen M, F, M, van der Meer J, W, M, White
    P, D, Is a Full Recovery Possible after Cognitive Behavioural Therapy for Chronic Fatigue Syndrome? Psychother Psychosom 2007;76:171-176

    2. “Both graded exercise therapy and cognitive behavior therapy assume that recovery from chronic fatigue syndrome is possible and convey this hope more or less explicitly to patients. Adaptive pacing therapy emphasizes that chronic fatigue syndrome is a chronic condition, to which the patient has to adapt.”
    Bleijenberg G, Knoop H “Chronic fatigue syndrome: where to PACE from here?” Lancet 2011; DOI: 10.1016/S0140-6736(11)60172-4.

    Like

  4. I’ll point out that one way around this problem is to require data release up front. F1000Research and some other journals do just that. At first I thought it sounded like a hassle for authors, but now I realize that it is actually a benefit for authors: among other things, it costs to keep data online, and it makes one curate the data better up front.

    Like

  5. This is a very helpful clear account. It is worth reviewing the last month’s editions of the Daily Telegrapgh as there have been at least three articles on the FOI and suggestions about the burden it is placing on public sector institutions etc. It struck me that were I in a position as a university employee that a decision on something had been made by more senior managers that they would then stick by their position and it would be worth more than my job to go against the decision and I would expect similar requests to be treated in the same manner even if they came from different people. However my perception of the fact that the position that has been taken on the foi requests not to provide the data has to be one of intentional cover up and therefore an even bigger distrust of the results of the study is held. I assume that if the data were clear cut analyses by others could quickly confirm this. If the data is not available for interrogation by others for whatever reason it is not unreasonable to suppose that there may be issues. This begs the question about the peer review process of articles in journals and recent debate in THES was that just too much volume and pressure for review of all articles to be effective. Hence a further argument for data transparency to enable robust academic challenge and debate.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s