The unfolding story of removal of data from a PLOS One article

  • mistakesA data set included in a 2015 PLOS One article has been removed, leaving the article no longer compliant with the PLOS data sharing policy.
  • The article reports secondary analyses from the FINE sister trial to PACE, but was published later than the PLOS One PACE paper and is subject to a stricter data sharing policy.
  • Removal of the data set was announced with a correction notice indicating the data set had been published “in error.”
  • The correction notice states that ethical permission (patient consent form) did not expressly allow data sharing.
  • Examination of the consent form indicates that it does not differ from many in use in the UK.
  • The correction was posted by PLOS One only a month after it had been received, substantially quicker than such corrections are typically processed.
  • The data set had been previously downloaded and used to demonstrate that outcome switching in PACE trial had been the basis for claims of positive findings, substantially affecting effect sizes and reversing conclusions in a now controversial Cochrane review.
  • The effective use by a nonacademic to counter misrepresentations in the Cochrane systematic review represents the kind of benefits anticipated for ready availability of data from clinical trials, but also the activity of “research parasites” condemned by foes of routine data sharing.
  • Allowing removal of the data set and posting of the correction raises concerns about a retreat from PLOS One published data sharing policies.
  • The author’s  making the data available to “qualified researchers” represents an exclusion not previously part of the published PLOS data sharing policies currently in effect.
  • The article is now in noncompliance with the PLOS data sharing policy that authors be the named individuals who are responsible for ensuring data access.
  • Broad issues are raised about the continued lack of transparency in negotiations between PLOS senior management and investigators in the UK, to the exclusion of other significant stakeholders, and allowing of substantial concessions to UK chronic fatigue syndrome investigators.
  • A substantial proportion of studies recently published or under review for publication in PLOS One have consent forms that do not expressly allow data sharing.
  • Presumably authors of a manuscript submitted today would simply include such a statement in order to sidestep the PLOS data sharing policy.
  • Does this represent the collapse of the PLOS data sharing policies with a loophole being granted through which most investigators could pass?
  • Despite the removal of the data set from the PLOS One article, it remains available on the Internet. Here‘s a link.
  • Three actions should occur immediately: (1) restoration of the data set to the article with an apology from the FINE investigators; (2) a full explanation from senior editor Iratxe Puebla as to how the stealth change to the PLOS One article so rapidly occurred; and (3) involvement of other stakeholders in the negotiations between PLOS One and King’s College London and Queen Mary University London.
  • Last week I requested a telephone conversation with Dr. Puebla and she indicated she would get back to me. But for now, academic editors, the pool of reviewers on which PLOS depends, and authors need to consider other options if the journal does not reassert a commitment to its previously published data sharing policies.

Commentary and documentation

The corrected PLOS One article from which the data were removed is here.

The original statement in the article concerning availability of data:

 “The authors have prepared a dataset that fulfills requirements in terms of anonymity and confidentiality of trial participants, and which contains only those variables which are relevant to the present study. Data are available as Supplementary Information”.

The corrected statement now reads

S1 Dataset was published in error. The error was corrected in the XML and PDF versions of this article on May 9, 2016. Please download this article again to view the correct version.”

maxresdefaultWhat does it mean that the data set was published in error? Was the “error” a matter of accidentally striking the wrong key on a computer keyboard? A mis-sent email? Saying a data set was published in error recalls “mistakes were made.”

The New York Times has called the phrase a “classic Washington linguistic construct.” Political scientist William Schneider suggested that this usage be referred to as the “past exonerative” tense,[1] and commentator William Safire has defined the phrase as “[a] passive-evasive way of acknowledging error while distancing the speaker from responsibility for it”.[2] A commentator at NPR declared this expression to be “the king of non-apologies“.[3] While perhaps most famous in politics, the phrase has also been used in business, sports, and entertainment.

life-in-hell-mistakes-were-madeManchester University quickly responded to inquiry whether the previously availability of the data in the PLOS One article represented a violation of the UK Data Privacy Act of 1998. MU offered the reassurance that the data set is “anonymised and contains no information which could identify an individual and so there is no need for us to report to the ICO [Information Commission Officer].”

The statement concerning Data Availability in the corrected article is now:

“Our ethical permission did not expressly permit us to share patient data, even anonymised patient data, in a public forum. Data will be made available to bona fide researchers on application to the principal investigator, Alison Wearden or the trial statistician, Graham Dunn. Alison Wearden can be contacted at: Alison.wearden@manchester.ac.uk. Graham Dunn can be contacted at: Graham.dunn@manchester.ac.uk.”

Journalist Leonid Schneider provided links to both the patient consent form from FINE and a comparison to what was used in PACE. He points out:

The original FINE patient consent form from 2004 is available here, it is also very similar to the patient consent form of PACE. The patients were not specifically deciding on allowing or prohibiting the open sharing of their anonymised patient data, but the form makes references to UK Data Protection Act from 1998, which may or may not prevent such data release.

The PLOS One data sharing policy in effect when the article was published is here

I asked a University of Pennsylvania bioethics professor, Jon Merz, JD, PHD for a comment on the consent forms and the PLOS one correction. He replied:

 If not prohibited, a la “we will never share your data, even if anonymized, with others.” then sharing is allowed, and should be encouraged.  The fact these authors admit sharing with others whom they deem “bona fide” researchers gives them unfettered discretion, which discretion has been abused.  By what standard do they decide?

Journalist Leonid Schneider reports the following statement from Alison Weardon:

We published the PLOS One paper on therapist effects and therapeutic alliance and provided a de-identified dataset containing the variables used in the analysis.

On 10th March, I received a freedom of information request for a copy of the patient consent form for the FINE trial. The request referred to the ongoing case relating to the PACE trial, and raised the issue of whether in fact we had been correct to make the data relating to the therapist paper open to the public. We did not request permission to do so in our trial consent form.

The dataset supplied to support the PLOS-One article was supplied in good faith and in the belief (still held) that no patient or therapist would be identifiable from it. The Freedom of Information request made me wonder if we had acted correctly, given our ethical permissions. In consultation with my co-authors and after discussing with various colleagues, I decided that it would be better to remove the dataset from public access (while still being prepared to supply it to bona-fide researchers). I wrote to PLOS-One on 18th April asking them whether it would be possible to do this.

The contents of the paper have not been retracted. The dataset has not been retracted. There is nothing wrong with either of them. The only issue is whether or not we were right in publishing this dataset given the consent that we had obtained from the trial participants“.

In the absence of other information, I think we can reasonably assume that the “various colleagues” included the PACE investigators.

In fending off Freedom of Information Act requests for sharing data, Peter White has repeatedly argued that it is a legitimate reason for withholding data if the data might be used to embarrass the investigators’ claims and reputations. The data from the PLOS One article was used effectively to discredit claims of the PACE investigators and raised concerns about conflicts of interest in a Cochrane systematic review in which they provided consultation.

See how one individual of this data allowed the unmasking of the effects of the day outcome switching in the PACE trial in a PubMed Commons comment by Sam Carter, Exploring changes to PACE trial outcome measures using anonymised data from the FINE trial.

I discuss the importance of Carter’s work in a blog post concerning the opening investigation of undeclared conflict of interest having produced a flawed Cochrane review in a dismissive response from the author to critics. Probing an untrustworthy Cochrane review of exercise for “chronic fatigue syndrome.”

As an academic editor of PLOS One, I am personally offended by senior management’s apparent acceptance of a restriction on data sharing to the vague category “bona fide researchers” and their encouraging further violations of data sharing policies by allowing the authors of the paper to decide themselves about release of their data on a case-by-case basis.

Peter White and the PACE investigators reject sharing data even with PhD’s who do not have formal academic appointments. Recently White made the argument that posting  on patient forums disqualified Dr. Wood from accessingthe PACE data [UK expert: AIDS data should not be shared until requesters shown to be HIV-

Many accomplished persons who do not have academic appointments effectively contribute to post publication peer review and to Cochrane reviews. Hilda Bastian is a great example. She also has delightfully funny blogs with PLOS. Hilda does not have PhD, but she played a crucial role in launching and sustaining PubMed Commons. Hilda works at the US National Library of Medicine, and has no  academic appointment. Would  she be turned down for access to the FINE data because she is a mere librarian, not a professor.

Graduate students, postdocs, and even patients effectively use PubMed Commons to comment on published work. They often use publicly available data sets from published studies PubMed Commons access only requires that someone can document that they have been an author on one of the 27 million entries in Pubmed, even a letter to the editor. I doubt this would be sufficient to qualify as an “bona fide researcher” especially with FINE authors as the judges.  They are closely aligned with PACE, their sister trial investigators. At this point, I don’t expect them to be detached, honest brokers of access to the data.

The PLOS management is aligning itself with Richard Horton, The Lancet editor who has such a contemptuous attitude to individuals outside of the Oxbridge network, as well as The New England Journal of Medicine Editor,  Jeff Drazen:

research parasites

PLOS One has also aligned itself in opposition to more progressive UK voices about data sharing like the present and former editors of BMJ. See

Richard Lehman’s Journal Review, Share data or be damned

Godlee, F. Call for greater involvement of patients.  BMJ 2015

Loder E, Groves T. The BMJ requires data sharing on request for all trials.

Richard Smith: QMUL and King’s college should release data from the PACE trial

Smith R and Roberts  Time for sharing data to become routine: the seven excuses for not doing so are all invalid F1000Research 2016, 5:781

Did the FINE authors get special treatment in the seemingly expedited handling of the correction?

A case can be made that they did.

I called the attention of the PLOS management to a 2015 article  from a Harvard Medical School professor that was accepted by a PLOS One Academic Editor at Harvard Medical School. The article had substantial undeclared conflicts of interest, was essentially a flawed experimercial for “integrative” medicine services offered by Harvard and violated PLOS data sharing policies by not making data available.

PLOS eventually responded to my complaint in February 2016, sharing a draft correction notice. It still has not appeared at the Journal site and these other serious issues remain unaddressed. Yet in the interim, the FINE authors were able to get their correction notice up and their actual article in less than a month.

For further background see

In the standoff over release of the PACE PLOS One trial data, has the journal just blinked?

What patients should require before consenting to participate in research… 

UK expert: AIDS data should not be shared until requesters shown to be HIV-

Recognizing when “protecting patient privacy” is mere excuse for not sharing data

Postscript

I am losing patience with PLOS. My confidence in their commitment to data sharing is faltering. I remain convinced that PLOS One can reestablish its place at the forefront of open access, transparency and reporting, and availability of data from the analysis. I’m frustrated with the opaque negotiations that are been going on with UK investigators, the lack of involvement of other stakeholders, and the concessions that are been granted.

if today

It may be too early to act, but is not too early to begin planning an action in which academic editors, the reviewers on whom they depend, and the authors who submit papers avoid involvement with PLOS One for a month. During that time, the promotional booths that PLOS sets up at various scientific meetings should be given wide berth. Or- may be walk up and say “What is the PLOS data sharing policy for someone from Arkansas, not Oxbridge?”

 

 

Advertisements

18 thoughts on “The unfolding story of removal of data from a PLOS One article

  1. This is the trial where the nurses running it famously called house and ged bound patients with a severe form of a neurological illness ‘bastards who do not want to get well’ because said patients were not recovering by ridiculous graded exercise.

    Like

    • Some background on this: the nurses were told that chronic fatigue syndrome is a mental disorder where the patients keep themselves deconditioned by an irrational fear of exercise, with gradual exposure to the source of fear (ie. exercise) being the cure. It appears that the differences in how patients see the illness versus how the nurses saw it was so great as to make conflict inevitable.

      Liked by 1 person

  2. The views that other analysis of trial data is theft (such as from the NEJM) shows a remarkable selfishness and I think is unethical behaviour. Patients agree to be experimented on not to further the careers of researchers but to help get new treatments for illnesses. To restrict data access is to break with the assumptions of patents that the data will be used to develop new treatments. To delay benefits for the sake of the publication records of those who collated data should be seen as unethical.

    Liked by 1 person

  3. THIS COMMENT WAS MADE “IN ERROR” AND HAS BEEN REMOVED.

    It turns out I didn’t have express written permission from myself to post it after all. Silly me. Don’t worry though, if you are a “qualified reader” then do get in touch, if I like the look of you, I’ll be happy to share with you.

    Like

  4. James wrote: “Last week I requested a telephone conversation with Dr. Puebla and she indicated she would get back to me.”

    Iratxe Puebla does not have a PhD. There exists, towards the best of my knowledge, also only one peer-reviewed paper with Iratxe Puebla as author. See http://www.ncbi.nlm.nih.gov/pubmed/14580793

    There is currently also no background information available for all members of the staff of the journal PLOS ONE, see http://journals.plos.org/plosone/s/staff-editors This was still the case on 8 April 2016, at least for Iratxe Puebla. I have a PDF with the text at https://www.plos.org/staff/iratxe-puebla/ downloaded on 8 April 2016. This url currently leads to ‘404: Page Not Found’.

    Such details with background information and/or a list with competing interests are (still) listed for for example all members of the Executive Team and the Board of Directors of publisher PLOS, see https://www.plos.org/people Such information is for example also listed for all members of the staff of the journal PLOS Medicine, see http://journals.plos.org/plosmedicine/s/staff-editors

    I fail to understand why this is currently not anymore the case for all members of the staff of the journal PLOS ONE.

    Like

  5. Considering the publication of the FINE data is being used as an argument in Alem Mathees PACE trial ICO case, the timing of them pulling this data from publication is looking quite transparently strategic.

    From documents obtained from the ICO by Valerie Eliot Smith, there are several references to FINE and the way it very clearly undermines arguments against PACE data sharing.
    The document is posted here:
    https://valerieeliotsmith.files.wordpress.com/2016/04/010-090216-r2-matthees_main_response.pdf

    Here is a relevant excerpt:
    “Again, the respected and popular PLOS journal publisher now requires publication of all data
    underlying a summary article, including individual patient data. The disputed information passes the
    revised data sharing policy used to publish individual patient data from the FINE trial. [27] FINE
    was the ‘sister’ trial to PACE and both were funded by the Medical Research Council (MRC). The
    timing, design, patients, outcomes, and interventions, were all similar.
    The disputed information contains 12 linked variables and no participant ID numbers, whereas the
    dataset released by FINE contains 18 linked variables including participant ID numbers. Both
    contain the same type of information about fatigue scores, physical function scores, whether they
    met a diagnostic criteria for the condition, and which specific group they were randomised to. Upon
    release of the dataset, PLOS issued this statement: “The authors have prepared a dataset that
    fulfills requirements in terms of anonymity and confidentiality of trial participants, and which
    contains only those variables which are relevant to the present study.”
    The above clearly contrasts to the approach of QMUL. By conducting a similar study to PACE and
    publishing in PLOS One with a liberal data sharing policy, Goldsmith et al.’s publication seriously
    undermines the assertions by QMUL that granting my FOIA request is a violation of anonymity,
    confidentiality, MRC data sharing guidelines, etc. QMUL’s far-fetched concerns have not prevented
    the FINE group from voluntarily providing open public access to individual patient data (FINE
    26. http://www.meassociation.org.uk/2016/02/me-association-writes-in-support-of-foi-request-relating-to-release-of-pace-trial-data-9-
    february-2016/
    27. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0144623
    12
    clearly do not believe that participants will be identified). It is difficult to see how the disputed
    information is a violation of patient privacy given the above. No FINE trial participant has, or
    unlikely ever will be identified from the individual patient data published.”

    Like

  6. If a lack explicit consent from participants for sharing annonymised data is seen as a justification for avoiding PLoS guidelines on data-sharing then a lot of researchers will be able to avoid making their data available. It is difficult to believe that the removal of this data is not related to the fight against the release of PACE data.

    Liked by 1 person

  7. > “Broad issues are raised about the continued lack of transparency in negotiations between PLOS senior management and investigators in the UK, to the exclusion of other significant stakeholders, and allowing of substantial concessions to UK chronic fatigue syndrome investigators.”

    This is a core issue. Once again it is all being done behind closed doors, without the involvement of the primary stakeholder: patients.

    Completely unacceptable, in any scientific field, let alone one as controversial and human centred as this. Nothing will make it more controversial than opaque deliberations and power games by those in authority.

    > “Data will be made available to bona fide researchers on application to the principal investigator, Alison Wearden or the trial statistician, Graham Dunn.”

    > “By what standard do they decide?”

    Exactly. Serious conflicts of interest all over the place here. I don’t believe for one second that people like Wearden and Dunn do not understand that. This is the data sharing arrangement you agree to when you don’t really want to share data.

    If they were genuine they would be asking for a properly independent decision making process to determine who gets access.

    > “In fending off Freedom of Information Act requests for sharing data, Peter White has repeatedly argued that it is a legitimate reason for withholding data if the data might be used to embarrass the investigators’ claims and reputations.”

    This is the worst excuse of the lot offered by the PACE/FINE crowd, and a big red warning flag about their real intentions in denying access to data. (I will give White a point here for honesty, even if inadvertent.)

    The hard truth is that the investigators have embarrassed themselves and trashed their own reputations with their shoddy work, and inaccurate and unethical claims about it. Their critics are just pointing all this out, as they should. That is how science works.

    All the more shame on the investigators, and the formal peer review process that has enabled their shenanigans, that those critics are not only proving consistently correct on critical points, but are also often not ‘qualified professional expert approved bona fide researchers’ to the bargain.

    Liked by 1 person

  8. There seem to be a double standard at PLOS One, they advertize themselves as open access, but twice now, they allow authors to no provide access to or withdraw data, those instances happening for ME, which is a field that desperately needs transparency and scrutiny of data when it comes to psych experiments coming from the UK.

    I thank Dr James Coyne for shining the light and blow the whistle here, because patients need to be protected from harm, and we know the PACE trial and Sister study FINE trials are both very controversial.

    PLOS One needs to stay true to who they are. Publish the data, or retract. Stand for good science.

    Like

  9. The more I look at the situation in the UK about improving the standards of research, the more despondent I become. The medical research council distributes some £700 million pounds as research grants and in keeping with the current fashion has a detailed set of guidelines and policies about what is required in order to qualify for funding. These were produced collaboratively with a number of other organisations, I suspect at considerable expense and include the requirement for preregistration, data to be stored in a particular way, this will facilitate sharing which is also a requirement. I suspect that researchers in medical specialities will be a fairly select group , many will be known to the awarding committees.
    I did ask them how the process would be policed, how would they know the grant would be used in the way suggested and what would happen if they refused to fulfil their agreements, their answer was, Oh we can’t do any of that, we leave that to the Universities. So basically they give away public money, often to people they know and rely on the organisations with the greatest vested interests to check on things.
    The policies look impressive and detailed but in reality are a waste of ink, its a form of meaningless advertising that they no commitment to, in the same way they have no commitment to improving the quality of the work they fund. The Lancet appears to have the same view and support the status quo over any quality improvements.
    PLOS has very much sold itself on taking the moral high-ground and insisting that many of the recommendations for improving research quality are followed in the papers it publishes. However the vested interests in the research establishment can be very powerful and despite the huge problems in the credibility and reliability of research, the so called international effort to improve quality is more a trendy fashion statement. While there are some examples of improvements in practice they are not overwhelming. Sadly it seems that PLOS are not quite as committed to honesty and openness as they claimed and have entered into the world of secrecy and covert politics. As PLOS was seen as revolutionary and people were and are keen to support its aims, I think their lack of credible action could effectively destroy it, it would be a betrayal of its followers.

    Like

  10. I decided to write to the Data editor at PLOS One to ask them if they had any comment to make on whether they saw any connection between the FINE trial correction and removal of dataset and the ongoing refusal by the sister trial, PACE. to share a similar anonymized dataset as per their contract with them. The reply I got didnt set my pulse racing…. “Thank you for getting in touch.

    The authors of the article contacted the editorial office about possible restrictions for the public availability of the data in relation to the information outlined in the consent form for the trial. The data was removed to avoid any possible breach in patient privacy. The PLOS ONE editors are actively following up with the authors regarding the dataset and any ethical requirements to allow the data to be shared.

    Best,” ……………………………………………….

    So I thought I would go back and be a little bit clearer. Currently awaiting a reply to this:

    “Many thanks for your response. Much appreciated. I understand the sensitivities around this issue but you haven’t addressed my query, I.e. can you comment on the connection between the correction of the FINE trial paper and the refusal by the PACE trial to share a similar dataset as per their original contract with you?

    On a point of information I would like to mention that there is a process of appeal currently underway at the FOI commission in London regarding refusal by the PACE trial authors to allow access to their anonymised data. The applicant has listed, as part of his appeal case, the fact that the FINE Trial had already made similar anonymised data available and therefore there could be no logical reason for the PACE authors to refuse the request based on privacy concerns. Therefore it would appear that there is indeed a connection and your comment on this would be much appreciated.

    Best regards”

    Like

  11. James Coyne wrote: “Three actions should occur immediately: (1) restoration of the data set to the article with an apology from the FINE investigators; (2) a full explanation from senior editor Iratxe Puebla as to how the stealth change to the PLOS One article so rapidly occurred; and (3) involvement of other stakeholders in the negotiations between PLOS One and King’s College London and Queen Mary University London.”

    James Coyne is totally right, but my experiences with getting feedback / explanations from Iratxe Puebla when it comes to tough items are until now very bad. Iratxe Puebla does not respond, and contacting her colleagues at publisher PLOS has no sense (auto-replies, or no response at all). This implies towards my opinion that “she indicated she would get back to me” does not exclude that James Coyne will need to wait to until 30 June 2045 before he will get a response.

    I am for example already waiting 308 days on getting from Iratxe Puebla the correspondence about a faulty paper. Please read https://scholarlyoa.com/2016/01/14/another-controversial-paper-from-fronters/#comment-405567 to get some insight in my experiences until now with Iratxe Puebla. There is until to day (29 May 2016) no follow up.

    Like

  12. Readers of this blog post who would like to get more insight in my experiences until now with high-ranked members of the staff of publisher PLOS, and in particular with Iratxe Puebla, might also be interested to read my lengthy opinion at https://scholarlyoa.com/2016/01/12/one-problem-with-the-scholarly-publishing-industry/#comment-407941

    These readers might also be interested to know that I received on 31 December 2015 an e-mail from Veronique Kiermer, the Executive Editor of PLOS. “Sent: Thursday, December 31, 2015 1:41 AM; Subject: re: Your correspondence to PLOS ONE editorial office. (….). We find your previous requests (referring to your emails from October) and your continuous suggestions of a conflict on the part of Iratxe Puebla or PLOS inappropriate. Sincerely, Veronique Kiermer”.

    There is until now (31 May 2016) no follow up, which implies that it is indeed not excluded that James Coyne will need to wait until 30 June 2045 before he will receive a response. I can therefore imagine myself very well that James Coyne wrote: “I am losing patience with PLOS.”

    Like

  13. I got a reply to my reply from PLOSOne “Thanks for your additional inquiry. PLOS ONE follows up on concerns about publications according to the specific circumstances for the individual article and situation arising; we are therefore considering the two cases that you comment on individually according to the conclusions drawn and the data specific to each article. Our follow-up is guided by consideration of the issues raised and the journal’s policies, and is independent from FOI processes.” Interesting though that the dataset has now been reinstated

    Liked by 1 person

  14. James wrote in this blog post (aired on of before 24 May 2016):
    * “last week I requested a telephone conversation with Dr. Puebla and she indicated she would get back to me.”
    * “three actions should occur immediately: […] (2) a full explanation from senior editor Iratxe Puebla as to how the stealth change to the PLOS One article so rapidly occurred.”
    Anyone any idea if there have already been these contacts between Iratxe Puebla and James Coyne?

    James wrote also in this blog post (aired on of before 24 May 2016): “three actions should occur immediately: […] (3) involvement of other stakeholders in the negotiations between PLOS One and King’s College London and Queen Mary University London.”
    Anyone any idea if this is already the case?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s