Breaking News: PLOS One issues Expression of Concern for PACE trial paper

experssion of concern

Expression of Concern: Adaptive Pacing, Cognitive Behaviour Therapy, Graded Exercise, and Specialist Medical Care for Chronic Fatigue Syndrome: A Cost-Effectiveness Analysis

Several readers have raised concerns about some of the analyses reported in the article and made requests for the data underlying this study.

The PLOS ONE policy governing the sharing of data that applies to articles submitted before March 3, 2014, requires that authors agree to make freely available any materials and data described in their publication that may be reasonably requested for the purpose of academic, non-commercial research (http://journals.plos.org/plosone/s/file?id=c4aa/PLOSONE_data_policy_before_2014March.pdf).

We assessed the concerns raised and the requests for data and we sought advice from two editorial board members. The advice we received was that the individual-level patient data for Tables 1,2,3,4 and 5 are necessary to replicate the cost-effectiveness analyses reported in the article. In line with the advice received, we contacted the authors to request the individual-level patient data for these tables. The authors raised concerns related to patient confidentiality and specifications under the consent sought from participants at the time of recruitment for the trial.

In consideration of the requirements for ethical oversight of data access that may apply to datasets involving human subjects, we contacted the authors and Queen Mary University of London, where the dataset is held, to request that steps be taken to develop a mechanism that would allow requests for data to be independently reviewed and the data released in accordance with our policy while respecting patient privacy. The authors and Queen Mary University of London shared the data policy in place at the institution, however we consider that aspects of the existing framework impose limitations and conditions not aligned with our editorial policy.

The authors have offered to release aggregated data from the study but have reiterated reservations about the public release of individual-level patient data. The journal policy does not require public release of anonymised patient-level data, but does require a suitable framework for data access for the purpose of academic, non-commercial research. While the release of summarized data does not fully comply with the journal requirements, we welcome that the authors are now willing to share summarized data, and we will provide this once it is made available to us.

In spite of requests to the authors and Queen Mary University of London, we have not yet received confirmation that an institutional process compatible with the existing PLOS data policy at the time has been developed or implemented for the independent evaluation of requests for data from this study. We conclude that the lack of resolution towards release of the dataset is not in line with the journal’s editorial policy and we are thus issuing this Expression of Concern to alert readers about the concerns raised about this article.

Statement from the authors

We disagree with the Expression of Concern about our health economic paper that PLOS ONE has issued and do not accept that it is justified. We believe that data should be made available and have shared data from the PACE trial with other researchers previously, in line with our data sharing policy. This is consistent with the data sharing policies of Queen Mary University of London, and the Medical Research Council, which funded the trial. The policy allows for the sharing of data with other researchers, so long as safeguards are agreed regarding confidentiality of the data and consent as specified by the Research Ethics Committee (REC). We have also pointed out to PLOS ONE that our policy includes an independent appeal process, if a request is declined, so this policy is consistent with the journal’s policy when the paper was published.

During negotiations with the journal over these matters, we have sought further guidance from the PACE trial REC. They have advised that public release, even of anonymised data, is not appropriate. As a consequence, we are unable to publish the individual patient data requested by the journal. However, we have offered to provide key summarised data, sufficient to provide an independent re-analysis of our main findings, so long as it is consistent with the REC decision, on the PLOS ONE website. As such we are surprised by and question the decision by the journal to issue this Expression of Concern.

Advertisements

17 thoughts on “Breaking News: PLOS One issues Expression of Concern for PACE trial paper

  1. This is very welcome. And thanks to Professor Coyne for his continuing work to hold the PACE investigators to account.
    I note the investigators claim in their response ‘that public release, even of anonymised data, is not appropriate’.
    This argument was addressed by the Information Tribunal (IT) in re Matthees. The IT was very clear: where data can be anonymized, there is no legal or ethical consideration which prevents release; that release is permitted by the consent forms; that there is a strong public interest in the release; that making data available advances legitimate scientific debate; and that the data should be released.
    The investigators should not be allowed to hide behind an argument which has been dismissed by the English and Welsh courts. PLOS One must continue to insist on the release of patient-level data. If the investigators do not provide these data, then the paper must be retracted.
    Trial decision here http://informationrights.decisions.tribunals.gov.uk/DBFiles/Decision/i1854/Queen%20Mary%20University%20of%20London%20EA-2015-0269%20(12-8-16).PDF

    Liked by 5 people

    • Quick off the mark, James, thank you, and well done John for saying what needs to be said by PLOSONE. I couldn’t agree more. I sincerely hope that the MRC – who are cited in the author’s reply – are busy updating their data sharing and ethical policies in light of the court’s and even this decision. But I guess only time will tell. In the meantime, I am grateful to those who are able to carry on the good fight while others like myself cannot. It will be interesting to see what you can make from the summary data they have said they will release.

      Liked by 2 people

    • Dear PLOS One editors,

      The PACE trial authors and QMUL have a long history of trying to prevent independent scrutiny of the PACE trial data.

      In 2016, one freedom of information request for a selection of anonymized patient-level data was successful, but only after a legal battle that was resolved by a tribunal which ruled in favor of releasing the data [1]. The tribunal dismissed QMUL’s argument that data could not be shared. An independent re-analysis of this data then showed that “the claim that patients can recover as a result of CBT and GET is not justified by the data, and is highly misleading to clinicians and patients considering these treatments” [2].

      An important aspect is that several PACE trial authors, including the principal investigator Peter White, have ties to the health insurance industry, which benefits from the PACE trial’s conclusions. Insurers can avoid long term payments to CFS patients if they can successfully argue that the patient doesn’t have a medical condition but rather a behavioural disorder that is reversible with CBT and GET. David Tuller has written on the PACE team’s work for insurance companies [3]. Unusually, the PACE trial was in part also funded by the UK Department for Work and Pensions (which also handles chronic illness related disability pensions). Mansel Aylward, who at the time occupied an important position in the Department for Work and Pensions sat on the PACE trial steering committee which approved the absurd changes to the main outcome and recovery analyses.

      The public interest in the PACE trial case is large: more than 100 scientists, clinicians, patient organizations signed a letter to the journal Psychological Medicine asking them to retract the misleading recovery claims [4]. A petition asking for retration of misleading claims made in the PACE trial collected 12000 signatures [5].

      The tribunal ruling has already made clear that there are no good reasons to withold anonymized patient-level data of this type. It is rather obvious that QMUL and the PACE authors are engaging in stalling tactics and have no intention to allow anyone to scrutinize this data because it will contradict their conclusions just like the data relating to recovery did. So please just get it over with and retract. Thank you.

      1. First Tier Tribunal: QMUL vs Information Comissioner and Matthees
      http://informationrights.decisions.tribunals.gov.uk/DBFiles/Decision/i1854/Queen%20Mary%20University%20of%20London%20EA-2015-0269%20(12-8-16).PDF

      2. Can patients with chronic fatigue syndrome really recover after graded exercise or cognitive behavioural therapy? A critical commentary and preliminary re-analysis of the PACE trial
      http://www.tandfonline.com/doi/abs/10.1080/21641846.2017.1259724?journalCode=rftg20

      3. Trial by error, Continued: PACE Team’s Work for Insurance Companies Is “Not Related” to PACE. Really?
      http://www.virology.ws/2015/11/17/trial-by-error-continued-pace-teams-work-for-insurance-companies-not-related-to-pace-really/

      4. An open letter to Psychological Medicine, again!
      http://www.virology.ws/2017/03/23/an-open-letter-to-psychological-medicine-again/

      5. Misleading PACE claims should be retracted
      https://my.meaction.net/petitions/pace-trial-needs-review-now

      Liked by 3 people

      • By long history of trying to prevent independent scrutiny of the PACE trial data, I’m referring to various freedom of information requests that were made. These were often rejected for reasons entirely unrelated to anonymization, therefore it appears that QMUL will use whatever excuse is good enough (I remember one time they claimed that in order to fulfill the request, they would have to hire and train a statistician). The reason cited for their refusal to share data seems to have shifted to risks related to de-anonymization when it became apparent that they would have to fight this out in tribunal.

        I understand that PLOS One needs to carefully consider all aspects, but please do not let the PACE authors stall this forever. They are not concerned about patients. If they were, they wouldn’t report highly misleading recovery figures. They are not concerned about de-anonymization. If they were, they would have consistently cited this as the reason data can’t be shared. FOI requests for PACE trial data are archived on http://www.whatdotheyknow.com

        Liked by 2 people

  2. Ah yes. The PACE data sharing policy. Which requires a pre-specified analysis to be done on the data, and states that ‘we prefer to be collaborating authors in the research, we may occasionally provide data without this’.
    Sharing data only with a selected group who will be nice to you isn’t really sharing.

    Liked by 2 people

  3. It would be interesting to know what exact advice the REC gave to QMUL and the exact information and claims that were made to the REC. But then the REC is not independent as it has been claimed that they approved protocol changes and so it would be embarrassing for them if they the release of data demonstrated more spin.

    They have already released some individual patient data in the original Lancet paper around SAE (in the appendix) and the information tribunal brushed aside their concerns about confidentiality. If the data is not anonymised then it would be illegal to share under their data sharing policy. If the data is anonymised then I see no issue and they have not been able to articulate one.

    Perhaps given the REC’s opinion they should look at other research that has issued patient level data and question those involved!

    Liked by 2 people

    • Yes, I wondered about how independent the REC could be at this point. No argument from the REC was provided to justify their failure to comply with PLoS policies, just an appeal to authority. A number of concerns have been raised about the conduct of this trial and apparent problems with oversight. The PACE researchers have stated that some of their protocol deviations had been approved, although have been less than clear on others. I think that we need more transparency not just on the results from the trial, but also on the systems and institutions that have played a role in slowing scrutiny of this work.

      An open letter, first sent in 2015, called for an independent investigation, and since then the PACE researchers have done nothing to lessen concern about their work: http://www.virology.ws/2016/02/10/open-letter-lancet-again/

      Liked by 1 person

      • My suspicion is that the protocol changes were approved by the approval of the statistical analysis plan. I don’t think that plan actually talks about the changes made to primary or secondary outcomes but it just changes them. Hence my question would be did they have explicit permission to change outcomes with associated reasons that were accepted or was it done implicitly by the acceptance of the SAP.

        The claimed reasons for protocol changes in papers are weak and should not stand up to scrutiny. So this leaves a question as to whether the changes were properly scrutinized. So far we have been blocked from seeing minutes of committees who have supposedly approved changes. Given the claims they make about the results I think we should say until evidence is provided that the claims were explicitly made then we assume they were not. Too me evidence would have to be the meeting minutes where approval occurred and included associated discussions.

        We should remember the Plos one paper deviates from the original protocol and as such the data provided needs to be sufficient to assess the differences and the effects of the changes.

        Like

  4. I imagine what the PACE Trial authors are really saying is: If we release anonymized patient data, you will see that we lied and based our careers on this lie and our entire lie is “too big to fail”. ~Anton Mayer https://twitter.com/CoyneoftheRealm/status/859407322870886400

    You will see that we used the Oxford criteria and the CDC 1994 criteria and were diagnosing chronic fatigue and NOT Chronic Fatigue Syndrome. Those that met CFS criteria got worse, could not complete GET/CBT, or just made no improvement. Those that improved were, in fact, suffering from CF which is due to many diseases, illnesses, and drug therapies, but CF is not the grossly misnamed disease CFS. https://voat.co/v/CFS/254402

    Our careers are on the line so you ain’t getting the data.

    Liked by 3 people

  5. When you publish a paper in a journal, the journal has a certain number of rules tyat you must comply to. These rules are clearly specified before or during submission process.

    What seems so weird to me is that the authors are saying to the journal tyat they (PLOS One) must comply to QMUL’s rules first.

    It’s not Kosher.

    I thank PLOS One for putting an expression of concern on this paper. Patients with ME from around the world have been suffering from the dubious antics from the authors of this paper and PACE trial.

    The Same expression of concern should have been stamped on the original PACE in The Lancet for a while now.

    Thank you to James Coyne for making my day today.

    Liked by 2 people

  6. After calling those suffering from the grossly misnamed disease Chronic Fatigue Syndrome the following: Militant Patients, Malingerers, having “false illness beliefs” and “vexatious”, and now “anti-science”, I am sure their career nightmare would be releasing patient-anonymized data from the PACE trial.

    They will never release the data and the only right thing to do is to retract this paper from PLoS One.

    The Lancet and Psychiatric Medicine should also retract the PACE trial paper.

    Like

  7. The Academic Editor of this paper is Pieter van Baal, Erasmus University Rotterdam, The Netherlands. See below for information (in Dutch) about his advice to the editors of PLOS about the sharing of the data. This information is copy/pasted from an e-mail which I received from him on 25 January 2016 and which I am hereby releasing to the readers of this blogpost.

    “—– Original Message —–; From: Pieter van Baal; To: Klaas van Dijk; Sent: Monday, January 25, 2016 10:50 AM; Subject: RE: Verzoek om een update over de stand van zaken over het verstrekken van alle ruwe data van pone.0040808

    Beste Klaas,

    Ik heb het verzoek van PloS One gekregen om nog eens goed naar de studie te kijken en mijn advies uit te brengen. Ik heb de studie opnieuw gelezen en ben tot de conclusie gekomen dat in mijn ogen de kosteneffectiviteitsstudie correct is uitgevoerd, maar dat als de trial niet correct is uitgevoerd alle resultaten (inclusief natuurlijk de kosteneffectiviteit) in twijfel moeten worden getrokken. Mijn advies aan PLos kwam er op neer dat het me verstandig lijkt om de data die nodig zijn om de kosteneffectiviteitsstudie te repliceren openbaar te maken (vanwege alle commotie maar ook omdat de studie publiek is gefinancierd). Ik ben wel bang dat het openbaar maken van de data geen helderheid kan geven over de vraag of de trial correct is uitgevoerd. PLoS is zich aan het beraden over volgende stappen.

    Vriendelijke groet, Pieter van Baal”

    Liked by 2 people

  8. Myself and Tom Kindlon wrote detailed comments published under the said PACE paper – I personally requested access to the PACE trial data almost 3 years ago – but was denied with no reason given. The PACE authors have hidden behind consent – yet shared their dataset with select colleagues. The PACE authors failed to uphold a fundamental principle of science – which is to allow others to challenge your findings but making your findings transparent. To the contrary, the PACE authors used every trick in the book to deny myself and others access to their data – until a brave patient from Australia, supported by others, challenged this closed door policy by taking his case to a Freedom of Information Tribunal. Even here the PACE authors circled the wagons and got Queen Mary University to payout over £200k on lawyers to try prevent this patient getting access to a tiny fraction of the PACE data.

    I totally agree with PLOS One, I salute them for standing up for science and their policy on data. Well done to Jim Coyne for taking up this issue with Plos One.

    Let PACE be an example for others to learn from.

    Liked by 3 people

  9. Oh happy day! One more chink in the bulwark these BPS researchers have built around their data to protect it from the eyes of others. Thank you, Dr. Coyne. Your efforts are greatly appreciated, as is the integrity of the PLOS ONE editors.

    Liked by 1 person

  10. Disingenuous. If patient level data was collected with patient anonymity required (as per the Declaration of Helsinki), pretty obviously any qualified researchers can’t make that patient level data public without breaching their own IRB requirements.

    Liked by 1 person

  11. So the staff of PLOS ONE received advices from 2 experts, one of them the Academic Editor of this paper, in which it is stated: “the individual-level patient data for Tables 1,2,3,4 and 5 are necessary to replicate the cost-effectiveness analyses reported in the article.”.

    The authors still refuse to release these data to the readers of this paper, and are thus still violating the guidelines in regard to the free access of the raw research data of this paper.

    I totally fail to understand why the staff of PLOS ONE has not decided to retract this paper.

    So the authors refuse to accept the guidelines in regard to free access for all to the raw research data, and 3th parties, for example James Coyne, are thus still unable to replicate / verify / validate / scrutinize etc. ‘the cost-effectiveness analyses reported in the article’. Excuse me very much my dear members of the staff of PLOS ONE, but that’s now how it works.

    James Coyne et al. want to ‘replicate the cost-effectiveness analyses reported in the article’ and you have still been unable to order the authors that they must provide James Coyne et al. the data which is needed to ‘replicate the cost-effectiveness analyses reported in the article’.

    Below an example from another journal, not member of COPE, which responded totally differently after a reader had contacted the corresponding author and at the same time the journal with some queries. This example landed recently in my e-mail box.

    I propose that the professional members of the staff of PLOS ONE reflect thoroughly on the acting of the editor-in-chief of this journal (all details are redacted). I emphasize that this journal is not a member of COPE.

    1. “On Wed, Feb 22, 2017 at 6:38 PM, [redacted, a reader] wrote: Dear [redacted], Could you be so kind as to please address my concerns with your Figure 1, published very recently in the special issue of [redacted]. If you could please provide the original figures in high resolution that were used for Fig. 1B and Fig. 1C to me and the [redacted] editors, that would be much appreciated. Also, I could not identify any [first name, last name] at the [redacted] Institute for [redacted], only [same first name, different last name] [redacted urls]. If you could kindly provide the professional website and email of your middle author, that would be important to assess his opinion about this figure.
    I look forward to hearing from you and your co-authors at the earliest. Sincerely, [redacted, a reader]”

    2. “On Sat, Feb 25, 2017 at 7:19 AM, [redacted] wrote: Dear Author, please response to the inquiry. It will let the journal to determine whether errata or retraction is needed. Regards, Editor-in-chief”

    3. “On 2 Mar 2017 15:31, [redacted] wrote: 1. Regarding my publication in [redacted]. 2. This work is my Ph.d work which i did in [redacted] institute [redacted]. 3. About [redacted] he has retired from his work during 2004 itself, he guided my work. No contact with him now, since now I am in [redacted]. 4. If you give your quire in detail about this matter, if possible I will give explanation for you.”

    4. “On Sat, Mar 4, 2017 at 8:40 PM, [redacted] wrote: Dear Dr [redacted], your reply means that the paper is submitted and revised without the consent of one listed author. Moreover, the figure in question is not clarified. As an editor-in-chief of [redacted], I decide that your paper will be retracted. [redacted] “

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s