In the standoff over release of the PACE PLOS One trial data, has the journal just blinked?

plos oneI just received (April 7, 2016) another communication from the Managing Editor of PLOS One reporting  an emerging position on my gaining access to the PACE trial data, promised to be available as a condition for publishing in PLOS One.

The negotiations are not over. But is there a signal that PLOS One is prepared to render meaningless their distinctive commitment to full data availability and sharing?

21925139-Various-hands-during-tug-war-on-white-backgrounds-Stock-PhotoAny retreat from full and unrestricted availability of data has implications for those of us who work for free as Academic Editors, those of us who submit our manuscripts to PLOS One rather than somewhere else because of its data sharing policy, and even those institutions who been willing to subsidize hefty publication costs because of the distinctive claim that data from published articles will be available to all.

If PLOS One abandons or even waters down its commitment to data sharing, will we abandon PLOS One?

 As I’ve stated before, despite being in one of thousands of PLOS One Academic Editors, I have no special insight or influence over the decision-making in this matter. Indeed, the PACE investigators have more insight into what is going on, not only because they are directly involved in the negotiations, but because maybe PLOS One could be granting them an extraordinary latitude in whether they can refuse or bureaucratize what had been my simple request for the data.

The e-mail

Dear Jim,

Thank you for the correspondence regarding the article by McCrone et al.

As you know we are actively following up on the request for data from the article. We sought advice from two editorial board members in order to establish what data we would expect the authors to share in the context of the current analysis, and we have followed up with the authors. The authors have raised concerns about the need to protect patient privacy as well as the specifications outlined in the consent obtained at the time of recruitment for the trial; our follow up with the authors is ongoing.

We plan to contact Queen Mary’s University of London to discuss our policy and position in relation to the sharing of data from this study.

I would like to take this opportunity to outline what we can and cannot engage in as part of our follow up as editors.

The policy that applies to the article indicates the following:

Publication is conditional upon the agreement of the authors to make freely available any materials and information described in their publication that may be reasonably requested by others for the purpose of academic, non-commercial research.

Availability of data and materials. PLOS is committed to ensuring the availability of data and materials that underpin any articles published in PLOS journals. PLOS’s ideal is to make all data relevant to a given article and all readily replaceable materials immediately available without restrictions (while not compromising confidentiality in the context of human-subject research).

The policy therefore outlines that data should be shared for the purposes of academic research and without compromising confidentiality in the context of human-subject research. Our goal is for authors to release as much data as possible, however the policy cannot supersede the requirements for ethical/data access oversight that may apply to the use of datasets involving human subjects.

It is not within our remit as editors to judge what the requirements for consent, patient privacy or data de-identification should be, such considerations are handled by institutional committees (such as ethics committees), which are equipped to assess and rule on such matters. In the context of our policy we will follow up on any and all requests for data underlying publications in the journal, but there may be situations where such follow up establishes that access to the data requires a process of evaluation by an ethics committee. If that is the case, our position is that the readers requesting the data should pursue such approval process in the context of a defined research proposal, in the same manner as the researchers who undertook the work described in the article did.

Commentary

In making this new statement available, I believe the scientific and larger communities have the right to know what is going on and should have the opportunity to comment. We are all stakeholders and although PLOS One does not apparently have a mechanism for soliciting such comments, I think we can be heard and influential.

Here is some preliminary observations concerning the communication from PLOS One

It’s reassuring that the journal reaffirms its commitment to making all data relevant to the article available, but it’s worrisome that PLOS One is legitimizing previously expressed objections of the PACE investigators.

Namely, sharing the data would be inconsistent with their purported commitment to protecting privacy and to honoring the conditions laid out in obtaining patient consent to participation in the trial.

In a decision last winter, the UK Information Commissioner’s (IC) rejected arguments from the PACE investigators that fears of breaching confidentiality because of the inability to anonymize data precluded making the data available. As we now know, anonymizing can be accomplished with readily available technologies.

The recruitment and consenting materials that I have reviewed do not mention any commitment to restricting access to anonymized data. Indeed, the Medical Research Council (MRC) funded a sister trial to PACE for which data has already been readily available. I would be curious to know if there are any relevant differences in recruitment and consent materials between the two trials.

Furthermore, in recent arguments for rejection of a request for data from a PhD physicist, the PACE investigators did not even mention anonymizing or any concern for keeping the promises provided in obtaining patient consent as reasons for denying her the data. A letter from Peter White to the Wall Street Journal acknowledged the PACE trial data can be effectively anonymized.

Finally, the PACE investigators have disclosed that they have already shared the data with researchers outside their group.

It’s not clear whether the requirement for review of a request by an ethics committee for data opens new loopholes for withholding the data. I’m concerned about a non sequitur being introduced here: that the application to the committee has to involve a pre-specified research plan.

Where did that requirement come from? The PACE investigators notoriously changed their endpoints from their protocol. This raises curiosity not only about what results would’ve been obtained with the original endpoints, but also about what may be uncovered by more exploratory, even forensic, analyses probing what might have led to the decision and what the PIs might be avoiding.

Having the data from published papers available allows independently checking the basis for the original investigators’ claims. This becomes particularly important – as in the case of the PACE trial – when investigators have conflicts of interest and when the conflicts of interest were not disclosed to the study participants.

However, another widely accepted rationale for data sharing is that clinical trials are expensive and impose considerable burdens on patients who participate in them. The best use of the public funds that supported trials and of the patients’ contribution is to make data available for exploring research questions that were not anticipated, without the added expense and patient burden of conducting a whole new trial. I anticipate some anomalies being uncovered that would best be examined in analyses explicitly labeled and interpreted as exploratory.

It’s a relief that the PLOS One administration seems to be avoiding any endorsement of the kinds of loyalty or character tests that were so pivotal in refusals to release the data to date. The PACE investigators may have turned my request for the data into a matter falling under the freedom of information act and rejected the request because my alleged vexatiousness and impure motives in wanting the data.

I’m sure that many of you are disappointed that after all this time, an email from the Managing Editor of PLOS One did not simply provide simple instructions how I and others could access the PACE trial data.

I had also assumed that my request would yield circumstances by which everyone could access the data. It now appears that anyone wanting to access the data, needs to anticipate long delays and a frustrating and potentially uncertain bureaucratic process.

Am I missing anything in this communication?

We need to keep in mind that we always have the option of demanding a retraction of the PLOS One article if the investigators set unreasonable conditions for independently scrutinizing their dubious claims.

And then there is the much-anticipated final decision on April 22, 2016 concerning Queen Mary University London appeal of an Information Commission decision requiring release of the PACE trial reported in The Lancet.

Advertisements

24 thoughts on “In the standoff over release of the PACE PLOS One trial data, has the journal just blinked?

  1. If they cannot meet the requirements for the journal shouldn’t the paper simply be retracted.

    What is more of an issue is that the PLOS one statement seems to imply they will do nothing to help get data published. Obviously the institutions involved and their ethics committees have a huge conflict of interest in that publication of data could lead to serious questions being asked about how the results were spun and how the institutions failed to institute a governance process. It all seems very Fifaish

    Liked by 2 people

  2. Excellent work, but a disappointing reply from the PLosOne editors.

    Since all are agreed that the data can be anonymized, the PIs seem to be arguing that they have a duty to protect the data itself. This claim cannot be allowed to stand.

    PIs cannot be allowed an eternal right to determine what use is made of the data. Public interest must lie in making the data available: to maximize the benefit of the trial participants’ time and effort; to gain best value for the trial funding; and in the overwhelming need for data to be available for scrutiny to test for scientific rigour.

    The PACE PIs must not be allowed to establish a precedent. If they are, any drug company or quack peddler will be able to publish results and then determine who gets to see their data and what can be done with it.

    Liked by 2 people

  3. I think that the general point, i.e. that the journal doesn’t decide exactly what data should be released and what the ethical concerns might be, is unobjectionable. But I do wonder why it was considered necessary to send it.

    The flip side of this, however, is that any group of researchers who have the favourable ear of their ethics committee can avoid compliance with the PLoS data sharing policy for ever. We know from, e.g., Leonid Schneider’s work that in many cases of bad science, the institution rallies round to protect its reputation. This certainly appears to be what QMUL is doing.

    There is also the question of what “the context of a defined research proposal” means. If I suspect that a set of results are too good to be true because the data may have been tampered with (note: I don’t think there is any such suspicion in the case of PACE, and I am certainly not making such a suggestion here), I shouldn’t have to state a priori hypotheses about exactly which variables may have been affected. So whatever my “defined research proposal” might have to contain, it should not limit my ability to explore the data in any way; any possibly confounding consequences of such exploration will be a matter for the reviewers when I write up my findings.

    But an alternative interpretation is (simply) that lawyers are involved in some way, and that this e-mail doesn’t actually reveal anything about what anyone at PLoS wants (or thinks is likely) to happen.

    In a better world, researchers would be required to deposit their data when they submit their article, on the basis that they have already withheld whatever they consider to be ethically necessary. Then, it would be up to the action editor to decide if those exclusions seem reasonable, and whether the data set without the excluded items is sufficient to allow reproducible science to be published. We’ve seen, time and again, that once an article is published, there is very little the journal can do, especially as long as retraction is portrayed as the nuclear option.

    Liked by 2 people

  4. James,

    Thank you. I appreciated this blog very much.

    I don’t think – and never have thought – that the data would be released unless a formal study had been designed, was ready to run the gammit of ethical approval, and whose author(s) would be willing to make room for one of the PACE authors to appear on the subsequent publication.

    Members of the public I don’t think will ever be granted access to data that is not already attached to published papers. And this to me – a layman – seems utterly wrong. The PACE authors were only able to carry out this study with public money. They designed the study and should have made allowances for public release of the data. It’s almost as if they had deliberately chosen to go down this route so as to give themselves an excuse.

    Now we have some serious researchers interested in the data I sincerely hope that one or more of them (including perhaps yourself) might be willing to consider a structured research proposal if indeed such a method is the only route to accessing this information.

    Of course we still have the tribunal at the end of this month, and PLOSONE doesn’t yet seem to be a dead-duck. But maybe it’s time to start thinking about a formal application – though quite how you’d go about such a thing or where you’d obtain funding, is I am afraid beyond me.

    Russ

    Liked by 2 people

    • I certainly did not have the expectation when I jumped into the fight over the PACE trial data, but I’m increasingly confident that we will prevail. If we don’t get the data, the PACE investigators will have demonstrated they have something to hide. If PLOS one cannot get us the PACE data, they will revise their procedures so this will not happen in the future. I can assure you that it has been a very costly struggle for PLOS. In my history with them is that they learn from their mistakes, perhaps more than almost any other journal.

      Liked by 3 people

  5. ” During the PACE trial the eligibility criteria, both subjective primary outcomes, and most of the recovery criteria were altered, creating an overlap of the eligibility and recovery criteria; consequently, 13% of patients were considered “recovered,” with respect to 1 or 2 primary outcomes, as soon as they entered the trial. In addition, 46% of patients reported an increase in ME/CFS symptoms, 31% reported musculoskeletal and 19% reported neurological adverse events. Therefore the proportion negatively affected by CBT and GET would be between 46% and 96%, most likely estimated at 74%”

    http://www.sciforschenonline.org/journals/neurology/JNNB-2-124.php

    PLOS One, Lancet, please do your bit just to cut the crap and demolish this wretched so-called research once and for all. You know it makes sense if you want to claim any credibility at all.
    Thanks, from one of very, very many extremely long-term sufferers of the horrible illness ME/CFS. Sorry I can’t put it all into convoluted scientific language right now, but I’m just a very sick person who has had enough of being sick for thirty years.

    Thanks, Prof Coyne, for your efforts.

    Liked by 1 person

    • I think that those figures on harms are misleading without comparison to the SMC only control group. The PACE researchers have promoted the ‘benefits’ of CBT/GET without comparison to the control group, but it’s still a bad idea to do the same thing back at them imo.

      Like

  6. Thank you so much for keeping this important conversation alive and lively. We live in the 21st Century now, and it is the Age of Open Information. We are currently riding the changing tides, and no matter how much the old guard may protest and resist, the wave always crashes to the shore. Those who frantically resist the inevitable risk getting pulled under.

    Mr. Coyne, keep strong as you lead the way for this sea-change in science, journalism, finance, politics, information–there will soon be no where left to hide out from our innate desire to collaborate, cooperate and innovate for the benefit of all.

    Liked by 2 people

  7. As I read this, if you contact your local IRB, tell them what you are going to do with the data, and what precautions you are going to use to ensure that subject confidentiality will not be breached, they would give IRB approval.

    That removes “patient confidentiality” as an issue that PLoS One need be concerned about.

    I have no idea how onerous this would be for you to do.

    Liked by 1 person

  8. That the excuses the PACE authors have come up with are so varied suggests that these are just made up on the spot whenever data is being asked for.

    At the very least it should now be obvious that PACE authors are hiding something. Maybe something that’s far worse than an ineffective treatment, such as harm caused by CBT and GET being obscured. A ME Association survey found that 74% of patients reported being made worse by GET.

    Liked by 2 people

  9. Professor Coyne,

    Thank you for continuing to pursue this so doggedly. I’m surprised that the PACE authors have chosen to go with the argument that the patient consent conditions prevent them from sharing their data – not just for the reasons you’ve given but also because the MRC had a data sharing policy in place at the time the PACE trial got underway, which PACE was covered by. This policy – which had been in development since 2002 and on the MRC website in draft form since early 2003 – clearly stated that the MRC expected the data from all research they funded to be made available to the scientific community in a timely fashion and without undue restrictions. Investigators were supposed to take this into account when planning and carrying out their research – and yet there’s no mention of data sharing in the PACE trial protocol, on the trial website or on the consent forms for participants.

    The PACE team’s current argument seems to be “We wilfully ignored the data sharing requirements placed on us by the organisation that funded our work, and as a consequence we now can’t share our data with you.” I hope that both PLOS One and the Information Commissioner recognise this line of reasoning for the self-serving drivel it clearly is.

    Liked by 1 person

  10. Some preliminary comments on the letter from PLOS ONE of 7 April 2016.

    (1): James wrote: “I just received (April 7, 2016) another communication from the Managing Editor of PLOS One”. https://www.plos.org/about/people/staff/ shows that there is only one Managing Editor at PLOS ONE. Her name is Iratxe Puebla. I therefore take is for granted that the letter is signed by Iratxe Puebla. Am I right?

    (2): The letter states: “We sought advice from two editorial board members in order to establish what data we would expect the authors to share in the context of the current analysis.” The letter does not list the names, experiences and backgrounds of these two persons (experts?) and the letter also does not provide any insight in the advise which was given by these two experts. Readers of this blog post might wonder why the letter of PLOS ONE does not provide insight in these topics.

    (3): The letter does not provide details about points of view of the Academic Editor of this paper. I had contacted Pieter van Balen, the Academic Editor of this paper, about these ongoing issues in December 2015. I got a very friendly response. I have contacted him again on 22 January 2016, soon after James Coyne had reported that lawyers of both parties (publisher PLOS versus the authors and/or their affiliations) got involved. I got once again a very friendly response from Pieter van Balen on 25 January 2016. He stated: “Mijn advies aan PLOS kwam er op neer dat het me verstandig lijkt om de data die nodig zijn om de kosteneffectiviteitsstudie te repliceren openbaar te maken (vanwege alle commotie maar ook omdat de studie publiek is gefinancierd).” So the Academic Editor of this paper has advised to publisher PLOS that all raw research data which are needed to replicate this study must be released to a public repository. The Academic Editor of this paper mentions two arguments for the public release of these raw research data: (a) a huge public interest; (b) public funding of the study.

    (4): The letter from the Managing Editor of PLOS ONE does not mention a time table / deadline. This implies that it is unsure if there will be a decision before let him say 30 June 2039. It is even not excluded that the lawyers of both parties are still negotiating with each other for the next 50 years. Readers of this blog post might wonder why there is no time table / deadline, which implies that it is not excluded that several years will pass before lawyers of both parties will have reached an agreement with each other, and/or that such an agreement will never be reached.

    Liked by 1 person

    • ” It is even not excluded that the lawyers of both parties are still negotiating with each other for the next 50 years. ”

      I think this is just what the PACE authors would love! There is a very large cohort of people with ME/CFS who became very ill in the early to mid 1980’s. We have borne the brunt of the hijacking of appropriate, biomedical research into and treatment of ME by the bunch of influential behavioural psychs who have profited at our expense and created the terrible conditions in which the only mainstream NHS “treatment”, CBT and GET, makes people iller.

      Now, at last, this appalling medical scandal has come to light beyond the sphere of people with ME/CFS , our carers and campaigners on our behalves. Now widespread respected clinicians, academics and statisticians see what has been going on for so long, it would be unsurprising if those most responsible for the wretched PACE trial and all its fall-out would like nothing more than that all of us inconvenient old-timers, who had the audacity to get ill ages ago and are still ill, would die out before the data came to light.

      Liked by 1 person

  11. From the UK please don’t give up James and all who are helping as we have never been so close to getting the truth before. Another trial to include 5000 young people is going to take place if the funding application is granted for it! It is along the same lines as PACE and Lightening Process called Magenta. We canon allow more young people to be put through adverse therapies.
    Please help to stop this?
    A student from QMUL has also asked for the PACE Trial data to be released in their January student magazine.

    Liked by 2 people

  12. As an M.E sufferer of 3 years (a relative newbie) the number of times that CBT and GET have been mentioned to me as a solution to my illness astounds me. The repercussions of this poor ” research” are huge. I hate to even think of the number of M.E sufferers who do not know the background to this and believe the advice of Doctors, thus worsening their condition. So a big, big thank you to you James Coyne for continuing to fight this for those who are unable to. Your tenacity and perseverance do you proud.

    Like

  13. No sharing -> retraction + big fat warning to never trust authors again!
    Its authors job to take care they can adhere to rules set fort by PLOS One.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s