A story in the feisty Bristol Cable revealed Bristol University ignores its responsibilities to publish results of clinical trials. A follow up story announced the university’s pledge to remedy the situation.
The investigation of hidden data involved drug and medical device trials and excluded psychotherapy or psychosocial intervention trials. But here are two trials for which publication is overdue.
One of the trials evaluates a quack treatment, which the Advertising Standards Authority (ASA) advises cannot make claims about health benefits. Its promoter was paid handsomely for the treatment provided in the trial. He capitalized heavily on the trial to advertise his treatment as being evaluated in a clinical trial. Was the trial simply a lucrative infomercial for his discredited product?
There is some evidence that these trials did not go as planned, but that is all the more reason to publish results so that we can understand what went wrong.
The situation is sufficiently problematic for me to recommend:
Patients should not enroll in psychotherapy trials associated with Bristol University until results of these trials are published.
Patients should be advised that the burden and inconvenience of participating in research is not warranted if the results remain unpublished. You are not obligated to participate in research. Don’t let anyone tell you differently.
A headline in the investigative journalism platform Bristol Cable read:
A joint investigation by TranspariMED and the Bristol Cable shows that the University of Bristol has not posted summary results for a single clinical trial onto registries. On the main European trial registry, the university is listed as the sponsor of 14 drug trials involving over 1,600 healthy volunteers and patients. Three trials that ended in 2009, 2011 and early 2016 have still not posted results there. Most of the remaining 11 trials also seem to have been completed, but are still listed as ‘ongoing’, suggesting that the university is not keeping its medical research records up to date.
On Clinicaltrials.gov, the main American registry, the university has registered 16 research projects; it has posted summary results for none of them.
A follow up headline read
A spokesperson for the university responded that Bristol had recently started adding results to the European registry. He also explained that until last year, “anyone could set up an account naming the University as the umbrella organisation” on Clinicaltrials.gov. The university has now assumed central control over those registry entries and is working to clean up its data, the spokesperson said.
The founders of the Bristol Cable told The Guardian: ‘We’re holding people to account for their actions’
The investigation was part of a larger international effort to uncover and correct a scandal: results of many drug trials are never published. Or published reports switch outcomes to make the manufacturers’ drugs appear more effective than they actually were in the trials.
Psychotherapy trials are excluded from these investigations. Yet, publishing results of psychotherapy trials consistent with the originally planned outcomes is undoubtedly less common because of lack regulatory oversight of psychotherapy trials. The confirmation bias in published of psychotherapy trials could even be stronger than with drug trials.
But we really cannot get a precise estimate of how many results of psychotherapy trials are being kept out of view. Many psychotherapy researchers do not even register their trials – or register only after they have already peeked at some of their results and can revise what were the originally intended outcomes being investigated.
This situation is unacceptable because it allows the promotion of ineffective and even harmful treatments.
Moreover, investigators have a moral and ethical responsibility to patients who consent to participate in clinical trials. In a trial comparing an intervention the investigators expect to be effective to a control group, half of the patients will not be randomized to the intervention. In many trials, this means the patients will be assigned to the no-treatment control group and get nothing. But these patients will be burdened with extra visits to complete the outcome assessments needed to evaluate the active treatment given to the patients in the other group.
Presumably, if the patients assigned to the control group are not simply confused or misinformed, they voluntarily continue to show up because they believe they are contributing to science and to knowledge that will help others in their predicament, if not themselves.
The first trial
The trial registration is ISRCTN72645894 DOI 10.1186/ISRCTN72645894
The overall aim of this study is to investigate the feasibility and acceptability of conducting a randomised controlled trial (RCT) to investigate the effectiveness and cost effectiveness of early intervention for chronic fatigue syndrome (CFS)/myalgic encephalomyelitis (ME) compared with standard medical care in primary care.
Early intervention for fatigue. The intervention is based on the principles of cognitive, behavioural and graded exercise and is delivered by a trained therapist in an individual face-to-face session with telephone follow-up sessions. Follow up length: 6 month(s); Study entry: single randomisation only
Primary outcome measure: Recruitment, adherence and follow up. Timepoint(s): 3 and 6 months
Secondary outcome measures: Not provided at time of registration
Note that the primary outcomes are not patient clinical outcomes, i.e., whether patients benefited from being assigned to active treatment versus control. The study was explicitly described as a feasibility and acceptability study.
Psychotherapy studies are commonly described in the literature as pilot and feasibility studies, mainly to escape the criticism that the study was small and underpowered. But, if so, effect sizes for patient outcomes should be presented because they are likely to be exaggerated and not generalizable to what would occur in a larger, better controlled study or in routine practice.
I wonder if the consent forms provided to patients indicated that the investigators were not interested in whether they benefited from treatment.
What results of a freedom of information act request revealed
Feasibility: We aimed to recruit 100 participants to this study, but were only able to recruit 44. In addition, 9 (32%) who should have received EI did not have all the sessions offered.
Acceptability: Although 12 of the participants told us that some of EI was useful, 11 of these participants also found some parts unacceptable. Four participants were still experiencing fatigue after completing EI and two felt they wanted more help at the end. Four of those we talked to did not find EI acceptable and withdrew. Reasons included: not liking the treatment model, or the way it was delivered.
Fatigue, pain, mood and work status were no different between the two groups at the end of the study however the study was not large enough to show an effect.
Our results show that this type of study is not feasible and EI was not acceptable to a lot of participants
It is a disservice to the patients who participated in the trial that there is no published acknowledgement that the study did not reach its accrual goal because patients did not wish to participate. Patients who consented to treatment did not generally find it helpful. Publication of this study could serve as a warning that most primary care patients suspected of having chronic fatigue syndrome (CFS)/myalgic encephalomyelitis (ME) are not interested in treatment based on the principles of cognitive, behavioural and graded exercise. The minority who consented to participate did not find the experience helpful.
The second trial: SMILE
The trial registration is http://www.isrctn.com/ISRCTN81456207
The study protocol:
Crawley E, Mills N, Hollingworth W, Deans Z, Sterne JA, Donovan JL, Beasant L, Montgomery A. Comparing specialist medical care with specialist medical care plus the Lightning Process® for chronic fatigue syndrome or myalgic encephalomyelitis (CFS/ME): study protocol for a randomised controlled trial (SMILE Trial). Trials. 2013 Dec 26;14(1):444.
An overlapping pilot study
This protocol is also cited for a pilot for which no full report was published, only a summary. The results of this pilot study resulted in the outcomes being switched for the main study.
Crawley E, Mills N, Beasant L, Johnson D, Collin SM, Deans Z, White K, Montgomery A. The feasibility and acceptability of conducting a trial of specialist medical care and the Lightning Process in children with chronic fatigue syndrome: feasibility randomized controlled trial (SMILE study). Trials. 2013 Dec 5;14(1):415.
Was this really a separate pilot study or an abortive failed start to the study described in the protocol? We can’t tell.
Among the sparse details from the pilot study:
Recruitment rates were lower than anticipated, meaning that recruitment continued for longer than planned. This was mainly due to an underestimate of the number of eligible patients at the start of the study (nearly half were not eligible). However, a recruitment rate of 35.9% does not preclude a full study.
In addition to the difficulty measuring change in school attendance for those transitioning from GCSEs to A levels, it may be a poor outcome measure for those who do not consider school attendance their primary goal. We suggest that a full study uses other primary outcomes, such as the SF-36 or the Chalder Fatigue Scale and uses school attendance as a secondary outcome.
So, at a time when other evaluations of interventions for ME/CFS were undergoing coordinated switching of outrcomes because of null findings from the FINE trial with the original scoring, the SMILE trial seems to have switched outcomes as well.
The treatment being evaluated in the main study is described as:
The Phil Parker Lightning Process® (LP) is a trademarked intervention that is used for a variety of conditions including CFS/ME. It has been developed from osteopathy, life coaching and neuro-linguistic programming. The intervention includes three group sessions on consecutive days where young people are taught skills that they can try out for themselves including looking at their sitting and standing posture. Families currently pay approximately £620 to attend the LP course.
Even though over 250 children and young people a year use the LP as an intervention for their CFS/ME, there are currently no reported studies investigating the effectiveness or possible side effects (for example serious adverse events) of the LP.
This is an exceedingly weak rationale for inflicting a quack treatment on unsuspecting children. Ethically, there must be a scientific rationale for exposing children to an unproven treatment that might be ineffective or even harmful.
The Advertising Standards Authority (ASA) guidance has ruled that LP cannot be advertised as a treatment:
To date, neither the ASA nor CAP [Committee of Advertising Practice] has seen robust evidence for the health benefits of LP. Advertisers should take care not to make implied claims about the health benefits of the three-day course and must not refer to conditions for which medical supervision should be sought.
Yet, the information sheet for the trial is describing it as a treatment to patients.
The materials patients must read as part of participating in the trial reek of pseudoscientific misinformation.
LP trains individuals to recognize when they are stimulating or triggering unhelpful physiological responses and to avoid these, using a set of standardized questions, new language patterns and physical movements with the aim of improving a more appropriate response to situations.
* Learn about the detailed science and research behind the Lightning Process and how it can help you resolve your issues.
* Start your training in recognising when you’re using your body, nervous system and specific language patterns in a damaging way
What if you could learn to reset your body’s health systems back to normal by using the well researched connection that exists between the brain and body?
The Lightning Process does this by teaching you how to spot when the PER is happening and how you can calm this response down, allowing your body to re-balance itself.
The Lightning Process will teach you how to use Neuroplasticity to break out of any destructive unconscious patterns that are keeping you stuck, and learn to use new, life and health enhancing ones instead.
The Lightning Process is a training programme which has had huge success with people who want to improve their health and wellbeing.
The intervention imposes strange demands on patients that encourage dishonesty, including in completing the outcome evaluations:
Responsibility of patients:
Believe that Lightning Process will heal you. Tell everyone that you have been healed. Perform magic rituals like standing in circles drawn on paper with positive Keywords stated on them. Learn to render short rhyme when you feel symptoms, no matter where you are, as many times as required for the symptoms to disappear. Speak only in positive terms and think only positive thoughts. If symptoms or negative thoughts come, you must stretch forth your arms with palms facing outward and shout “Stop!” You are solely responsible for ME. You can choose to have ME. But you are free to choose a life without ME if you wish. If the method does not work, it is you who are doing something wrong.
Bristol University at first rejected a freedom of information act requests for how much developers of the treatment were paid but on appeal revealed mean cost of a course for trial participants was £567. This means £14,000 wasted on a junk intervention in the pilot study and much more was paid for the full trial.
Special thanks to John Peters for his assistance and his excellent detective work and use of the Freedom of Information Act.
See his blog posts here:
THE SMILE TRIAL (part 1): Why the trial should never have been allowed in the first place.
THE SMILE TRIAL (part 2): A trial so flawed as to be worthless.
THE SMILE TRIAL (part 3): The only one to benefit is Parker.
And also my post