Coyne of the Realm on the Daily Show and teaching scientific writing

During the recent Democratic National Convention in Philadelphia, I was fortunate to get a ticket to a filming of the Daily Show. I came away with some great insights for teaching my scientific writing course.

14232402_10211124730580979_880852685714409702_nThe Daily Show audience was lead into the auditorium at Annenberg Center hours before the show would begin. Eventually a staffer came out and announced that we were going to be the laugh track for the show. We would be expected to show great enthusiasm on cue. She engaged the audience with a calculated abrasiveness. For instance, she said that some members of the audience might have small gifts for the host Trevor Noah. The staff would later collect them. But if anybody tried to give Trevor a sketch that they had drawn, the staff would ridicule it in the back room without passing it on to him.

Being part of a laugh track is no fun. For many of us, the idea of the laugh track conjures up an obviously canned audience response stuck into sitcoms on TV decades ago. Laugh tracks have mostly fallen out of fashion. Fortunately, this experience was different.

To warm us up, a comedian came out and casually engaged the audience. He would pick out seemingly random people and ask their names and from where they came. He would then instigate a lively exchange, gently poking fun at them and involving the rest of the audience.

At one point, he asked a guy if he was married to the woman sitting next to him. The man replied “no, we’re just on a first date.” The comedian asked the audience to encourage the guy to ask the woman to marry him. They did so with great excitement, after comedian whipped them up and egged them on a bit.

Much of the banter was simply silly and would have fallen flat in other circumstances. But the comedian managed to provoke an increasingly energized and enthusiastic response from the audience in the back-and-forth exchanges.

The comedian orchestrated a few practice greetings of the star Trevor Noah before he arrived, with the audience encouraged to take to their feet, cheer, and wave their hands over their heads and clap. I was amused to see Vincent Price, the University Pennsylvania Provost, sitting a few seats away, jump up and down like a hungry trained seal working for a fish from his trainer.

The audience was eventually deemed ready to actually greet Trevor. He dashed out onto the stage and adeptly fielded deliberately odd questions from the audience before the start of the actual filming of the show.

When the show was closer to starting, a large, mysterious director who had mostly stayed in the shadows, emerged to signal the audience to arise. It did so with genuine excitement and apparent spontaneity.

After filming the first segment of the show, the director and some staff reviewed it and were dissatisfied. They required second takes of some interactions with the audience. We complied and I doubted the later TV audience would know what they saw was restaged.

In the second segment, Leon Panetta was given a rousing welcome from the audience as he came onto the stage. Panetta was first the Chief of CIA and then Secretary of Defense under Obama. He was well prepared for being poked by Trevor and much funnier than I would’ve guessed.

But the director halted the taping because the audience was apparently off in its timing. So we provided Take 2 of Panetta’s entrance with more excitement.

For the finale of the show, the audience needed to serve as a wildly cheering backdrop for Trevor Noah’s exit. This too required a second take. Someone sent me a screenshot from the video available on the web in which I can clearly be seen doing my part.

me on daily show

Coyne of the Realm teaching the scientific writing class

Some readers may have attended one of my workshops on “Writing high impact papers and what to do when your manuscript gets rejected.” I’ve been teaching the workshop for years around the world, most recently at the European Health Psychology Conference in Aberdeen, Scotland. Before that was Melbourne. Lund, Sweden is upcoming in November, Amsterdam and Valencia are after the first of the year.

melbourne ad

Poster from Writing Workshop at World Congress Cognitive and Behavioural Therapies, Melbourne, July 2016

A takeaway message of  the workshop is that in the current competitive environment, doing solid science and writing in proper English is insufficient to get your work the attention it deserves. Up to 40% of all manuscripts in some fields are summarily sent back without review to authors, often without anyone, even the editor having read them.

So, it’s incumbent on authors to make a compelling pitch for their paper, using three tools that they have: the title, abstract, and their cover letter. Each of these three is usually written only as a quick afterthought, after the manuscript is otherwise ready for submission. That could be a fatal mistake for any hopes of the manuscript getting out for review.

having to sell yourselfWhat, I have to stoop to marketing my work? How dreadful. Academia often attracts people who want simply do their science, not engage in public relations or self-promoting. They could be disappointed when this is not enough to get published in the particular journals where they would get the recognition they crave.

There may be some self-selection into academia, but are also some strong cultural differences, with some Europeans and Asians loathe to emulate Americans, whom they see as loud, brash, overbearing and unduly confident and impressed with themselves. My Dutch audiences, particularly, recall having been raised with an attitude of “kop boven het tall corn flower cut2maaiveld uitsteken– roughly translated as ‘the tall cornflower gets cut.’

Maybe, but like the audience on the Daily Show, you can get worked up for a role you have to play, particularly when you don’t take yourself too seriously. I know, getting your manuscript out for review and then into print is serious business. But take it all as dead serious play and it will hurt less when things don’t go as planned. Pump up for Take 2. Take the promotion of your manuscript as an orchestrated performance.

My workshops give participants background on the current publishing environment as well as strategies for to identify and reach the readership which will be most appreciative of their work. I rely on workshop slide presentations that have been continuously refined over the years and I have didactics to go with them. But what is most distinctive and engaging is a lot of back and forth interaction with the audience. I term it call and response. The workshop participants pitch me their soundbites, rough sketches of their ideas, and elevator pitches.  They get my response and we go back and forth with escalating focus and contagious enthusiasm.

Influenced by my having a bit part on the Daily Show, I deliberately shifted my presentation style in Aberdeen, relying much less on my slides and much more on spontaneous interaction with the audience. In the afternoon, I ignored the slides except to consult them at break to see if I had missed anything.

Taking a tip from the comedian, I started with a casual interaction with the audience, questioning them about who they were, what their stage of their career, their where they were from and their experiences publishing papers or trying to. I had no illusions of being Trevor Noah or his warm up comedian, but I got the need to loosen up the workshop participants.

Then I launched into individual call and response  interactions with them. I am sure many participants came away thinking that they had not known how excited they could be about their work, once was put into better focus. Hopefully their enthusiasm will prove contagious when it comes to submitting their papers and getting them reviewed.

I’m careful not to embarrass or confront participants in the workshop about it, but I have come to strongly suspect that many junior and even senior authors submit their manuscripts without giving much thought to why an editor should be interested in it or get excited about it.

 Capturing this for web-based courses

 I now realize that the web-based courses that I am planning need to have the sriracha sauce of some of these live exchanges. So…

My film crew and I are setting up some 90 minute sessions at colleges and universities in the larger Philadelphia area. For a limited time, we will come for free and I will instigate this kind of back and forth, call and response sequences to be captured on the film. I expect to have a good time, and I expect that the audiences will too. Maybe they’ll even be more likely to get the papers out for review

If you’re lucky, I’ll see you there. Otherwise, you can go to my CoyneoftheRealm website and sign up to receive a free 15 minute sampling of the course that will soon be coming available.

 

 

 

Outgoing Vice President of Dutch Academy of Science warns of sharing data with “your enemies”

On one of his last days in office before being retired, Dr. Jos van der Meer, the Vice President of the the Royal Netherlands Academy of Arts and Sciences warned of the reputational damage risked by scientists who share their data outside their circle of close colleagues.

Dr. van der Meer made his remarks to an interviewer from the Dutch daily de Volkskrant :

“Put everything on the internet, there is always someone who discovers something. You will find in every study a mistake. It will go to details and I predict that it be enlarged in order to send the entire study to the dump. ” For the availability of data from clinical trials, there is much to do. Van der Meer knows. “There is a widespread view that conditions should be attached.” Van der Meer said that the research leaders shared indeed in recent years at the request data with colleagues. “But to give all data to your enemies.” Enemies? “Yes, there are people who have apparently interest to label this as a bad study. It is a personal vendetta against two renowned scientists. ” Apparently there is a setting error in your brain and you can turn that back with this treatment Emeritus Professor Jos van der Meer, expert on CFS Positive effect Psychiatrists Sir Simon Wessely and Peter White have become the face of the wrong turn taken by science according to patients. And that’s really strange, says Van der Meer, because the effect of behavioral therapy has been much more researched, and the conclusions were unanimous. His own research in the Nijmegen Radboud University Nijmegen Medical Centre a few years earlier than the British study also found positive impact of behavioral therapy. That that this treatment works does not mean the disease is between the ears, Van der Meer stressed. “Apparently there is a setting error in your brain and you can turn it back with the treatment.”

Dr. van der Meer’s opinions about routine data sharing are quite at odds with those of the Dutch government and scientific community. Like much of the rest of the world, the Dutch have embraced the necessity of data sharing in order for scientists and the public who fund science to get the full benefit of research.

His remarks were timed to give last-minute support to his long friend and ally Peter White in the UK. White is one of the PACE investigators facing an order from the UK Lower Tribunal to release the data from the controversial PACE trial. The widely criticized study evaluated exercise therapy and cognitive behavioral therapy (CBT) for chronic fatigue syndrome.

Apparently Dr. van der Meer was unaware that Peter White’s co-investigator Michael Sharpe had already just disclosed that results of the PACE trial were substantially poorer when evaluated with the originally declared scoring of primary outcome variables. The unanticipated revelation demonstrated the need to share the PACE data for independent reevaluation, particularly given the clinical and public health importance that has been attached to it.

Dr. van der Meer also took advantage of the occasion to spin the interpretation of CBT research for which he was senior author.

Prins, J. B., Bleijenberg, G., Bazelmans, E., Elving, L. D., de Boo, T. M., Severens, J. L., … & Van der Meer, J. W. (2001). Cognitive behaviour therapy for chronic fatigue syndrome: a multicentre randomised controlled trial. The Lancet, 357(9259), 841-847.

He failed to disclose that a decade later release of the objective performance data from his study had revealed that CBT did not affect objective performance measures. See PubMeds Commons comment here by Tom Kindlon. Only subjective self-report data showed evidence of any improvement, which are quite vulnerable to uncontrolled nonspecific influences in an unblinded study. If these data had been available earlier to independent researchers, considerable misrepresentation by the original investigators could be countered.

Dr. van der Meer also has made claims that CBT restores prefrontal cortical brain volume in patients with chronic fatigue.

De Lange, F. P., Koers, A., Kalkman, J. S., Bleijenberg, G., Hagoort, P., Van der Meer, J. W., & Toni, I. (2008). Increase in prefrontal cortical volume following cognitive behavioural therapy in patients with chronic fatigue syndrome. Brain, 131(8), 2172-2180.

The uncontrolled, grossly underpowered study has received withering criticism [ 1 2 ]. I would wager that these results would not be replicated in an adequately powered, more appropriately designed, preregistered and transparently reported study.

Dr. van der Meer also misrepresented evaluations of CBT for chronic fatigue syndrome conducted by those not directly involved in the studies who did not share the original investigators’ conflicts of interest. Contradicting his claim of uniformly positive results, the July 2016 updated review provided by the US Agency for Healthcare Research and Quality concludes there is “low strength of evidence that CBT improves fatigue” and “low strength of evidence that quality of life is not impacted by CBT”

A true fossil, after his retirement, Dr. van der Meer will be on permanent display next to the bugs in amber at the Naturalis Biodiversity Center in Leiden.

Chronisch vermoeiden op zoek naar erkenning [Literally “Chronically fatigued looking for recognition] Here is a link to the actual article.

 The article is in Dutch and below is a rough translation made by a colleague [corrections welcomed].

Chronically fatigued looking for recognition. A major study of chronic fatigue syndrome in 2011 led to much controversy. Now the fire flares up again after a remarkable statement of the British court. By Ellen de Visser September 6, 2016

The controversy between patients and scientists on chronic fatigue has reached a peak after a remarkable court ruling. A British court has determined that all the data from a study of treatments for the disease should be released to an Australian patient. The university has yet to end in question this week to appeal. Patients have since the publication of that study, five years ago in The Lancet, serious doubts about the claims. The so-called PACE study, the largest study ever done on the effect of treatment for chronic fatigue, showed that patients can recover from the disease by two types of therapy: an exercise and a form of behavior therapy. contempt The results received worldwide attention. For chronic fatigue syndrome (CFS), from which millions of people worldwide suffer, for a long time there was no cure. The disease is characterized by extreme fatigue, but also muscle and joint pain and memory problems.

“Chronically tired? Get up and move, the newspapers headed. Patients felt insulted by that message, they regarded the findings as a lack of recognition. They assume that their disease is not psychological but has a physical cause, and set their sights on the two British psychiatrists who suggested otherwise. Remarkably, in recent years there was more and more support from scientists around the world who openly voiced strong criticism of the study design, statistical analysis of the data and the interpretation of the data. The study could obviously not be blinded, so patients knew which treatment they received, and halfway they were told in a newsletter which treatments were a government commission favorite. The result was based purely on subjective data. All the objective data, for example the outcome of a walk test, could not support the findings. Criteria for participation in the study would have been dubious. Forty professors recently wrote an open letter to the Lancet in which they insist that the results should be reviewed by independent scientists. The study shows, according to them “major flaws” and thus there are “serious concerns” about the reliability of the conclusions. Good science does not need protection Slogan activists Withholding data Attempts to obtain the underlying data, encountered long time opposition by the researchers involved and the university, Queen Mary University London.

The Australian CFS patient Alem Matthees continued until the British personal data authority agreed with him a year ago. The university appealed to the court, but drew the short straw. The university believes that the release of the data damages the relationship of trust with the patients because privacy has been promised to them. But the patient data are anonymous, according to the court it is not are likely that will lead to problems. Scientists found that the withholding of data at a time of open access cannot be accepted. “Good science does not need protection,” that was the slogan of the activists during the trial.

But Emeritus Professor Jos van der Meer, expert in the field of CFS, warns of the consequences of disclosure of research data. “Put everything on the internet, there is always someone who discovers something. You will find in every study a mistake. It will go to details and I predict that it be enlarged in order to send the entire study to the dump. ” On the availability of data from clinical trials is much to do, Van der Meer knows. “There is a widespread view that conditions should be attached.” Van der Meer said that the research leaders shared indeed in recent years at the request data with colleagues. “But to give all data to your enemies.” Enemies? “Yes, there are people who have apparently interest to label this as a bad study. It is a personal vendetta against two renowned scientists. ” Apparently there is a setting error in your brain and you can turn that back with this treatment Emeritus Professor Jos van der Meer, expert on CFS Positive effect Psychiatrists Sir Simon Wessely and Peter White have become the face of the wrong turn taken by science according to patients. And that’s really strange, says Van der Meer, because the effect of behavioral therapy has been much more researched, and the conclusions were unanimous. His own research in the Nijmegen Radboud University Nijmegen Medical Centre a few years earlier than the British study also found positive impact of behavioral therapy. That that this treatment works does not mean the disease is between the ears, Van der Meer stressed. “Apparently there is a setting error in your brain and you can turn it back with the treatment.”

London University [Queen Mary London] has slightly more than one week to appeal, Alem Matthees let us know via email. The chance of this is small: the case can only go to a higher court if there has been “an error of law” and the law is misapplied. Once the data to Matthees released, anyone can retrieve them. “I am convinced that the researchers did not cheat (note from me: he uses the strange word gefoezeld)” Van der Meer said. Again no access to research data. The American professor James Coyne, affiliated with the University of Groningen, has requested information from the PACE study. He requested last year, researchers at King’s College London to provide him data from a follow-up study published in 2012 in Plos One, the cost-effectiveness of the treatments in question. The university refuses inspection. The letter of reply, which Coyne has put on his blog, the university wrote that Coyne has “no serious reasons” that he only wants to use the data for a polemic and ‘reputational damage’ threat. That refusal has once again led to angry reactions from fellow scientists. “If the university is looking for reputational damage,” twittered the British professor Chris Chambers, “then it is enough to clog research.”

Clinical psychologists’ divided loyalties in providing psychotherapy to clients who feel coerced to participate

Psychologists face ethical dilemmas in conflicts between their loyalty to their clients versus to those who employ the psychologists to provide therapy under conditions in which clients perceive or actually face coercion to participate in therapy as a condition for receiving benefits.

  • Are professional organizations entering into troubled waters when they collaborate with institutions intent on restricting social benefits to the unemployed and disabled?
  •  The Mental Health Foundation (MHF) believes that UK clinical psychologists are insensitive to these ethical conflicts.
  •  An exchange on these issues between the MHF and a group representing clinical psychologists in the UK that notably included the British Psychological Society ended unsatisfactorily.

The Mental Health Foundation charged:

We believe your collusion with the government is now threatening to undermine the ethical integrity of the “psy” professions among service user/survivors and professionals.

Further:

It is not possible to consider this issue without considering the context of sanctions, cuts and persecution which is endemic in the current system. You fail to acknowledge that attending this proposed therapy may not be explicitly linked to conditions/sanctions but will feel so for many of its prospective clients who are on benefits. There is a structural power imbalance between job centre employees and those on benefits. With their income under threat, those on benefits will be especially susceptible to cues, suggestibility and positive reinforcement when attending job centres. Many on benefits have experienced oppressive power relations for much of their lives. Saying no in relation to an apparent free choice in this context is hugely difficult, especially when saying no has uncertain consequences.

And:

We are concerned that under-qualified and inexperienced staff, such as job centre coaches, will be in a position to make referrals to Health and Work programs. This is exacerbated by the fact that referrals are likely to be to IAPT workers, many of whom themselves lack in-depth training and experience of severe mental health issues. Inappropriate referrals are increasingly likely in a target-driven culture.

IAPT Refers to the Improving Access to Psychological Therapies  Program. My recent Mind the Brain blog post –  A skeptical look at The Lancet behavioural activation versus CBT for depression (COBRA) study criticized an article by 14 mental health investigators which claimed that less trained clinicians could provide a simplified behaviour activation therapy with no loss in effectiveness over trained clinical psychologists providing cognitive behaviour therapy for depression. I argued that the clinical trial reported in the article was rigged to demonstrate that less trained clinicians were more cost-effective than fully trained clinical psychologists.

The MWF acknowledged that professional organizations have gone on record as opposing coercion to participate in psychotherapy. However, MWF accused the professional organizations of ignoring the genuine threat to clients.

What is the Mental Wealth Foundation?

Mental Wealth Foundation is a broad, inclusive coalition of professional, grassroots, academic and survivor campaigns and movements. We bear collective witness and support collective action in response to the destructive impact of the new paradigm in health, social care, welfare and employment. We oppose the individualisation and medicalisation of the social, political and material causes of hardship and distress, which are increasing as a result of austerity cuts to services and welfare and the unjust shift of responsibility onto people on low incomes and welfare benefits. Our recent conference focused on Welfare Reforms and Mental Health, Resisting the Impact of Sanctions, Assessments and Psychological Coercion.

The MWF lists its member organizations as

  • Mental Health Resistance Network
  • Disabled People Against Cuts
  • Recovery in the Bin
  • Boycott Workfare
  • The Survivors Trust
  • Alliance for Counselling and Psychotherapy
  • College of Psychoanalysts
  • Psychotherapists and Counsellors for Social Responsibility
  • Psychologists Against Austerity
  • Critical Mental Health Nurses’ Network
  • Free Psychotherapy Network
  • Psychotherapists and Counsellors Union
  • Social Work Action Network (Mental Health Charter)
  • National Unemployed Workers Combine
  • Merseyside County Association of Trades Union Councils
  • Scottish Unemployed Workers’ Network

combatworkfareThe three-part exchange is reproduced below. It starts with a letter of concern from the MWF coalition. A response follows from a group of psychological organizations including the BPS. The third part of the exchange is an expression of dissatisfaction from the MWF of the insensitivity in the response from the psychological organizations and their failure to set up a meeting.

Open letter to therapy’s professional bodies on the psychocompulsion of welfare claimants – from the Mental Wealth Foundation

 21st March 2016

To:

British Association for Behavioural and Cognitive Psychotherapies

British Association for Counselling and Psychotherapy

British Psychoanalytic Council

British Psychological Society

United Kingdom Council for Psychotherapy

Professional bodies scrutinise Government ‘therapists in job centres’ plans

We write in response to your joint public statement of 7 March 2016 outlining the outcome of your meetings with the Government’s new Joint Health and Work Unit and your scrutiny of the Government’s plans to place therapists in job centres [1].

There is no indication that any consultation has taken place with members of your organisations with knowledge of these matters nor with service users, clients and their representative organisations. This lack of consultation and opportunity for wider reflection has contributed to your organisations departing from your own ethical structures and frameworks, and being seen as agents of harmful government policy [2]. It is by now generally accepted that the consequences of the DWP and government policy in this area are far reaching for physically and mentally disabled people on social security benefits. Inexplicably your organisations’ scrutiny of government plans has failed to recognise this.

The joining of Government Health and Work Departments is not helpful, and current DWP policy intended to reduce the socio-economic causes of mental illness to the one simple fact of unemployment is clinically and intellectually ridiculous. The resulting policy promoting work as cure, which your organisations are now supporting, is offensive and dangerous. It is wrong for therapy organisations to buy into the unthinking praise for ‘work’ that often forms part of the rhetoric of governments.

While for some clients improving employment prospects may be an objective, for many others this is not the case and may be profoundly damaging. Indeed, for some people, their mental health problems may have begun because of work e.g. through experiences of bullying in the workplace. This one size fits all approach is simplistic. Premature return to work can result in loss of confidence and relapses affecting future ability to get back to work. This can also lead to prolonged periods without benefits and no income [3].

You state that plans must be aimed at improving mental health and wellbeing rather than as a means of getting people back to work. These are not the aims and objectives being expressed by the people who are implementing the programme right now, involving targeting ‘hard to help’ clients who are likely to be people with enduring physical or mental health difficulties. For example in the Islington pilot project Councillor Richard Watts has stated, “We think there is much more that health services can do to promote the idea of employment for people with health conditions.” In the Islington CCG Commissioners’ report in November calling for employment services in GP surgeries to reach ‘hard to help’ claimants, they state that, “to improve the system we need to…maximise the contribution of all local services to boost employment, making it a priority for health, housing, social care and training. We need to open up how we talk to people about employment, including asking healthcare professionals to have conversations about work with patients, as part of their recovery. We need to give professionals the information and tools to help them to do this.” [4]. For all clients, establishing a trusting relationship is the first priority, involving respecting their current needs, perspectives and autonomy.

Jobs advertised on the BACP website in November 2015 have the explicit aim of getting clients back to work and engaging with employment services e.g. “your role will include: producing tailored health action plans for each client, focusing on improving their health and moving them closer to work…generate health and wellbeing referrals to ensure continued engagement with employment advisers” [5]. Similarly G4S advertise jobs for BABCP accredited CBT practitioners with job roles including: “Targeted on the level, number and effectiveness of interventions in re-engaging Customers and Customer progression into work” [6].

We respectfully submit that information about these jobs was known to all of your organisations when you issued your joint statement. This inconsistency is seriously misleading.

We are glad that you oppose conditionality, coercion and sanctions. Clearly such punitive measures have no place in the therapeutic relationship. We fail to share your reassurance from the government that these measures will not be pursued against clients. DWP have repeatedly claimed that sanctions are a last resort and only happen in a tiny minority of cases. The reality is that millions of people have been sanctioned. In the twelve months to September 2015 alone, over 350,000 ESA and JSA claimants were sanctioned [7]. In the Employment Support Allowance Work Related Activity Group the majority of sanctions were of people who have been placed in the group specifically because they are experiencing mental health issues and research shows that benefit sanctions on people with mental health problems has increased by 600% [8].

It is not possible to consider this issue without considering the context of sanctions, cuts and persecution which is endemic in the current system. You fail to acknowledge that attending this proposed therapy may not be explicitly linked to conditions/sanctions but will feel so for many of its prospective clients who are on benefits. There is a structural power imbalance between job centre employees and those on benefits. With their income under threat, those on benefits will be especially susceptible to cues, suggestibility and positive reinforcement when attending job centres. Many on benefits have experienced oppressive power relations for much of their lives. Saying no in relation to an apparent free choice in this context is hugely difficult, especially when saying no has uncertain consequences [9] .

Attempts to coerce people into work are detrimental not only to their health but to their safety and, in many cases, present a risk to life. The extreme fear and distress caused by the current welfare reforms, including changes in disability benefits and the new Work Capability Assessments, is widely reported including instances that have led to suicide [10]. Therapy alongside this coercive system breaches the ethical principle of non-maleficence.

You state that there must be choice as to location of therapy. There is a clear danger in putting DWP representatives into GP surgeries, community centres and food banks that are seen as safe havens for people on low incomes and benefits. The presence of DWP compromises this. DWP/Maximus workers in the GP surgery, with access to medical records, will serve as a deterrent to people visiting their family doctor. The model currently in use in Islington allows Remploy/Maximus workers to access and write into GP records; this jeopardises any commitment to client privacy and confidentiality [11].

The choice of method of therapy is an illusion and therapists of all modalities are subject to the stresses of an unjust target driven culture [12]. We are concerned that under-qualified and inexperienced staff, such as job centre coaches, will be in a position to make referrals to Health and Work programs. This is exacerbated by the fact that referrals are likely to be to IAPT workers, many of whom themselves lack in-depth training and experience of severe mental health issues. Inappropriate referrals are increasingly likely in a target-driven culture.

We are not reassured that the feasibility trials planned by the government will contribute to knowledge and understanding and are not reassured by your echoing what government is saying. Instead you and government must listen to the voices of survivors who describe the reality of government plans on their lives and are fighting for services with a vision of humanity beyond work [13].

It is clear from your public statement that you have failed to critically examine and scrutinise the ongoing activities of the Government Joint Work and Health Unit. We call on you to cease your engagement with this unit and instead hold a national stakeholder event which is guaranteed to involve the participation of representative organisations for service users and therapists with direct knowledge of the area, as well as professional bodies like yours. There should be no government involvement in such an event. From it, a representative group can be selected that will adequately represent the views of service users and therapists to the appropriate government departments as well as to the opposition.

References

  1. http://www.bacp.co.uk/media/index.php?newsId=3906
  2. https://www.opendemocracy.net/ournhs/dr-lynne-friedli-robert-stern/why-we-re-opposed-to-jobs-on-prescription
  3. https://www.morningstaronline.co.uk/a-a3e3-Joblessness-branded-a-mental-illness#.VuKWT4SFDzI
  4. http://www.islingtonccg.nhs.uk/Downloads/CCG/BoardPapers/20151111/4.3%20Health%20and%20Employment%20Programme.pdf
  5.  ‘Mental Health Advisor – Job Details’ Retrieved from http://www.bacp.co.uk/jobs/jobs.php November 21st, 2015. Available athttps://www.dropbox.com/s/a6p9mod1jb08dne/Mental%20Health%20Advisor%20-%20Job%20Details.docx?dl=0
  6.  http://careers.g4s.com/jobs/Cognitive-Behavioural-Therapist_58526/6 crisis
  7. https://www.gov.uk/government/statistics/jobseekers-allowance-and-employment-and-support-allowance-sanctions-decisions-made-to-september-2015
  8. http://www.independent.co.uk/news/uk/politics/benefit-sanctions-against-people-with-mental-health-problems-up-by-600-per-cent-a6731971.html
  9. For a fuller discussion of these issues, see http://mh.bmj.com/content/41/1/40.
  10. http://jech.bmj.com/content/early/2015/10/26/jech-2015-206209.full
  11. Para 4.3 http://democracy.islington.gov.uk/documents/s6740/Health%20and%20work%20-%20HWB%20update%20Jan%202016%20final.pdf
  12. http://www.pulsetoday.co.uk/home/finance-and-practice-life-news/gps-told-to-inform-patients-dwp-will-obtain-their-fit-note-records/20030820.article.http://files-eu.clickdimensions.com/hscicgovuk-amnje/files/emed3dpnlettertogppracticesv1.0.pdf?_cldee=Y29yYWwuam9uZXNAbmhzLm5ldA%3d%3d&urlid\
  13. http://www.theguardian.com/healthcare-network/2016/feb/17/were-not-surprised-half-our-psychologist-colleagues-are-depressed
  14. http://recoveryinthebin.org/2016/03/10/welfare-reforms-and-mental-health-resisting-sanctions-assessments-and-psychological-coercion-by-denise-mckenna-mental-health-resistance-network-mhr

Letter from psy-organisations to MWF, dated 24th March 2016:

Dear members of the Mental Wealth Foundation,

Thank you for your letter,

It might be useful if we started by clarifying a number of points that we have made repeatedly to the Department for Work and Pensions. We do not believe that anyone should be coerced into therapy and would denounce any coercion or sanctioning in relation to ‘job centre therapy’. As a result of ethical concerns raised last summer around coercion and sanctioning in relation to ‘job centre therapy’ we, as professional psychological therapy organisations, immediately contacted the Department for Work and Pensions. We were also acutely aware of, and remain acutely aware of, the wider context of sanctions and cuts.

Work is not always good for people’s mental health and wellbeing and we too recognize that poor quality, stressful and insecure employment can be detrimental and profoundly damaging. We also do not believe that employment should be viewed as a universally beneficial health outcome. At the same time, however, good employment can help people, by adding security and purpose to people’s lives, thus enhancing their mental well being. There should indeed not be a ‘one size fits all’ approach and we do not support a policy of work as cure.

We share the view that clients should have their current needs, perspective and autonomy respected but we also make it clear that we do not believe the role of therapists should be to get people back to work. Instead, we believe that the role of the therapist should always be to work with clients to help them resolve their mental health issues. We also do not believe people should receive psychological therapy in an environment where they feel uncomfortable and we do not believe client’s privacy should be compromised.

It is nevertheless important to recognise that access to psychological therapy remains restricted and people are suffering needlessly because of this. It is also the case that rates of mental health problems among people who are unemployed remain unacceptably high. There are likely to be a multitude of reasons for this, from loss of sense of purpose, to the stresses of the sanctions regime – and we wish to see all of these factors tackled. Improving voluntary access to psychological therapy for jobcentre clients is therefore a policy which deserves proper consideration and as organisations which represent psychological therapists, we recognise that the provision of appropriate, voluntary, therapy can play a role in alleviating distress.

The DWP has repeatedly told us that there will be no coercion involved or sanctioning of clients who do not wish to enter psychological therapy. It has also said that clients will have
a choice of where they can access their therapy. We have also secured a promise of a thorough evaluation of their planned small-scale co-location feasibility trial, which should provide a clear indication of whether people’s health and wellbeing is genuinely being improved in both the short and the long term. We suggest it is prudent to review these evaluations before prejudging a project that could provide genuine help to jobcentre clients who experience issues with their mental health.

We also note that your letter conflates different projects. Our organisations are concerned with the provision of therapy to jobcentre clients. We are mindful that other projects have
been initiated in regard to the co-location of work coaches in GPs surgeries, and that other organisations have engaged in scrutiny of these projects. We do not believe we are the most appropriate organisations to comment on those proposals, given they do not involve therapeutic coaches or psychological therapy – which is why our joint response made no mention of those plans.

We would also ask you to note that any past job adverts for positions outside of our organisations should in no way be regarded as the official opinion of any of our organisations individually or collectively.

Yours sincerely,

The British Association for Behavioural and Cognitive Psychotherapies
The British Association for Counselling and Psychotherapy
The British Psychoanalytic Council
The British Psychological Society
The UK Council for Psychotherapy

The MWF replied on 12 April 2016, raising further issues and proposing a meeting for further dialogue. At the time of writing, this and other requests to meet face-to-face have gone unacknowledged. 

From:

Mental Wealth Foundation

Mental Health Resistance Network; Disabled People Against Cuts; Recovery in the Bin; Boycott Workfare; The Survivors Trust; Alliance for Counselling and Psychotherapy; College of Psychoanalysts; Psychotherapists and Counsellors for Social Responsibility; Psychologists Against Austerity; Free Psychotherapy Network; Psychotherapists and Counsellors Union; Critical Mental Health Nurses’ Network; Social Work Action Network (Mental Health Charter); National Unemployed Workers Combine; Merseyside County Association of Trades Union Councils; Scottish Unemployed Workers’ Network; National Health Action Party

To:

British Association for Behavioural and Cognitive Psychotherapies; British Association for Counselling and Psychotherapy; British Psychoanalytic Council; British Psychological Society; United Kingdom Council for Psychotherapy

Professional bodies scrutinise Government ‘therapists in job centres’ plans

Thank you for your response of 24 March 2016. We would like to reiterate our invitation to a dialogue around these issues. We are a unique alliance of 17 diverse organisations, representing clients, therapists, campaigners and academics with a unique breadth and depth of perspective. We would very much like to share this wealth of experience and expertise with you; and to understand your position better. We think your members would expect you to meet with us, given the very diverse range of organisations that have united in order to communicate with you.

We appreciate your assurances that you oppose as unethical any coercion or sanctioning connected with psychological therapy’s contribution to the government’s workfare programmes. We also welcome your statement that “[you] do not believe the role of therapists should be to get people back to work”, and that the therapeutic value of employment is conditional both on individual circumstance and the nature of any particular employment and its environment.

At the same time, we are aware that as members of the New Savoy Partnership four of your organisations have welcomed recent workfare proposals, have initiated collaboration between DoH and DWP on Health and Work pilots, and have invited Lord Freud as Minister of Welfare Reform to open a number of your recent annual conferences. Your ‘Joint Pledge on Welfare’  states:

“We welcome the opportunity the Work Programme provides to support more people with mental health conditions into appropriate and sustainable employment.  Specifically, we will develop our expertise to help people with mental health conditions find, enter and remain in employment.” (emphasis added)

We cannot see how your position differs in any significant aspect from that of the DWP and you seem to have allowed yourselves to be drawn into becoming active partners in the government’s workfare policies. We believe – and we know many of your members agree – that the “psy” professions are being let down by our professional bodies going along with government aims and schemes, instead of using their professional knowledge together with service user experience to influence the proper provision of therapeutic services in more appropriate settings, for the benefit of clients and the profession. Clearly your views do in fact differ significantly from ours. Isn’t this something better explored at a meeting?

Over recent months, dozens of adverts for DWP financed mental health advisor and employment coach posts have invited applications from people accredited by you, including as we have already pointed out, those in Therapy Today. These job descriptions are explicit that the role is to get people off benefits and into work. Our concern remains that no action has been taken to inform or protect your members or their clients about involvement in work of this nature which breaches ethical practice.

The top-down nature of policymaking causes alienation and distrust of government workfare policies with a reliance on expert think-tank research, “evidence-based” reports, and a reluctance to engage in any real collaboration with either service users or practitioners.   We are offering you an opportunity to do something different. We believe it is within your role and responsibility, as national professional bodies representing psychological therapies, counselling and psychotherapy in the UK, to hold open an independent arena of public debate on issues of national policy.

We believe your collusion with the government is now threatening to undermine the ethical integrity of the “psy” professions among service user/survivors and professionals. A year ago, our joint letter to the Guardian signed by 440 psychologists, psychotherapists and counsellors raised the alarm about the probability of mandatory psychological therapy coming into being through the co-location of IAPT workers in Job Centres. A year on, the development of Health and Work projects under the DWP’s workfare banner is promoting more, not less fear, demoralisation, and distrust among the members of the organisations the MWF umbrella represents.

As survivors and witnesses of the impact of these developments, we are compelled to act against welfare to work policies and associated government-sanctioned psychological coercion which harms service users and professionals alike. You didn’t reply to our call to cease engagement with the Government Joint Work and Health Unit, and to hold a national event where all stakeholders views can be heard. We would like our campaign to include open debate with the professional bodies whose interests must surely include upholding the ethical values of psychological practice, for the sake of all.

We look forward to your response to our offer.

 

 

 

 

 

Is the Science Media Centre briefing on cognitive behaviour therapy trustworthy?

science-media-centre-welcome

The UK Science Media Centre (SMC) offered to journalists (and any citizen scientists too busy or too lacking in confidence to make their own critical appraisals) a briefing on cognitive behavior therapy .

I offer a quick critical appraisal of the “three top experts” advertised as giving an honest appraisal.

I conclude that what would be offered is highly biased. Any organization making claims that such an evaluation is trustworthy is itself untrustworthy. Please read on and decide if you agree with me.

Meta-lesson: When you encounter a source that screams to you that it is trustworthy, maybe you should be particularly skeptical and simply move on to alternative sources.

The announcement

September 2, 2016

CBT – does it really work?

Journalists came along to the Science Media Centre to discuss issues such as:

  • What exactly is CBT and how does it work? Is it just talking?
  • Where do we have robust evidence of CBT working really well?
  • What about issues like placebos and double-blind RCTs – does it matter if we don’t have the same level of rigour as for drugs?
  • Are there concerns about CBT being overused or misused? Can it cause harm? How do we ensure practitioners have sufficient training in its use?
  • How cost-effective is CBT? Are we using it enough?
  • Can we see a future where psychological therapies replace pharmacological ones for many mental illnesses?

The SMC invited in three top experts to give an honest appraisal of the evidence, to discuss how CBT fits into the wider picture and to explain the pros and cons of this therapy.

Speakers:

Prof. Rona Moss-Morris, Professor of Psychology as Applied to Medicine, King’s College London’s Institute of Psychiatry, Psychology & Neuroscience

Prof. Michael Sharpe, Professor of Psychological Medicine Research, University of Oxford

Prof. Dame Til Wykes, Professor of Clinical Psychology and Rehabilitation, King’s College London’s Institute of Psychiatry, Psychology & Neuroscience

Prof  Til Wykes

I have learned to be  skeptical of UK expertise offered with Dame or Sir in the title. So, having been declared a vexatious American, I’ll simply call her Professor Wykes. But given she is deemed a Dame, I will do her the honor of starting with her first.

Professor Wykes has written an Evidence-based Mental Health commentary that repeatedly screams “Take me seriously.”

 Wykes T. Cognitive-behaviour therapy and schizophrenia. Evidence Based Mental Health. 2014 Aug 1;17(3):67-8.

She attests to the quality of the evidence that she considered:

So is CBTp effective?

All the meta-analyses described here were carried out rigorously. They allow us to draw a picture of a therapy that developed from its roots in medication-resistant positive symptoms to the wealth of new targets and then to the more radical—the prevention of psychosis. Only one recent meta-analysis does not show beneficial effects after accounting for methodology. This odd one out is odd for very good reasons and provides us with a backdrop to understand the treatments required in the future.

For instance in a paradigm changing study for people with continuing symptoms but who refuse medication, Morrison et al19 demonstrated successful treatment by CBTp. This is the first time such an RCT has been carried out and the authors are to be commended on their rigour and particularly on reporting adverse events (2 in the CBTp group vs 6 in the control group). The study challenges the belief that CBTp is only an adjunct to medication treatment in more chronic populations—although we cannot yet conclude that CBTp should be recommended until we have more and larger trials.

I have written extensively about the Morrison et al study in blog posts  [1, 2] and in a letter to The Lancet in which I complained about its quality and the study having been registered after it was already well in progress.

The Morrison et al. study:

  • Lacked an adequate comparison control group.
  • Claimed to be evaluating CBT for psychotic patients refusing medication, but in the course of the psychological treatment, most patients received some antipsychotic medication.
  • Had more authors than patients maintained in follow-up.
  • Relied on voodoo statistics that ignored the large attrition from an already small sample in arriving at a positive conclusion.

I could go on but I think you get the picture of what I think of this study.

Keith Laws also delivered devastating blog posts about the study (such as 1 and  2) with excellent accompanying music selections. With Peter McKenna, Professor Laws also published a  letter in the Lancet. Professor Laws and his colleagues  also authored the meta-analysis that Wyke dismissed as “the odd one out” that “is odd.”

Despite Wyke’s evaluation, Professor Laws’ meta-analysis is much more consistent with what has been offered by the Cochrane collaboration. He and his colleagues delivered a devastating reply to Wykes that leaves the Dame looking rather silly and ill-informed:

Keith R Laws, KR,  McKenna, PJ Jauhar, S The odd one out. Evidence Based Mental Health.

They say:

 Professor Wykes discusses the effectiveness of CBT, contrasting the positive findings of her and co-workers’ meta-analysis in 2007 (Wykes et al, 2007) with our recent finding, in a meta-analysis using similar inclusion criteria, of small effects on positive symptoms that became nonsignificant in blind studies (Jauhar et al, 2014). As well as these two ‘comprehensive’ meta-analyses, she considers three recent selective meta- analyses by: Burns et al (2014), van der Gaag et al (2014) and Turner et al (2014). Wykes makes the point that our meta-analysis differs in its conclusions from these others. We would like to clarify why this may be the case, specifically with regard to inclusion/exclusion criteria and the reporting of effect sizes

It is not immediately apparent why Wykes rejects the obvious explanation for why our meta-analysis had weaker findings than her similarly comprehensive meta-analysis – more recent studies over the intervening years have largely been negative. This interpretation is supported by the fact that only 1 of the 11 studies included in our meta- analysis since Wykes et al 2007, has documented a significant impact of CBT on positive symptoms. Moreover, in the one significant study (van der Gaag et al, 2011) the authors noted that blinding was compromised. It seems inherently unlikely that CBT would somehow work better in treatment- resistant than treatment-responsive patients. To maintain that CBT works against delusions and hallucinations requires rejection of the finding from a comprehensive meta-analysis of little effect against positive symptoms in favour of those of a much smaller one, which employed rather convoluted inclusion and exclusion criteria.

understand-coverProfessor Wykes is on Twitter as @TilWykes, where she lavishly praises a volume she co-authored with other members of the British Psychological Society, Understanding Psychosis and Schizophrenia.

I have offered to debate the authors of this book, but they have consistently refused. In their absence, I have presented detailed critiques in blog posts [1, 2 3] and what I think is at least an entertaining slideshow from an invited presentation at the Royal Edinburgh.

 Prof. Michael Sharpe

 Professor Sharpe has been getting a lot of media attention as one of the authors of The Lancet PACE trial of cognitive behavior therapy and graded exercise for chronic fatigue syndrome. He and his co-authors claimed that would be damaging to their reputations to release the data in response to a Freedom of Information Act request. I did not make my request for that data under the FOIA. I took advantage of the authors having published another study in PLOS One, where acceptance was contingent on the data being available. The authors still have not complied, but they unkindly  labeled me as vexatious for having made the request

While praising the strength of the results and winning praises from the likes of Professor Sir Simon Wessely, these authors and their universities spent over £250,000 resisting the release of their data. Just days before the data were released as ordered, Professor Sharpe publicly conceded that rates of improvement were substantially lower than reported in The Lancet. The discrepancy was due to the outcomes having been switched in The Lancet.

Among the many other problems in the credibility faced by Professor Sharpe and his co-authors is that the prestigious  US Agency for Healthcare Research & Quality has declared Sharpe et al’s case identification used in the PACE trial deficient.

Using the Oxford case definition results in a high risk of including patients who may have an alternate fatiguing illness or whose illness resolves spontaneously with time. In light of this, we recommended in our report that future intervention studies use a single agreed upon case definition, other than the Oxford (Sharpe, 1991) case definition.

Prof. Rona Moss-Morris

 In blogging about the PACE trial, I have said numerous things about Prof. Rona Moss-Morris, not all of them flattering, but never as unflattering as what I have said about Professors Wyke and Sharpe. Here I will only say that Professor Moss-Morris was brought in by the Science Media Centre to offer an expert reaction to the follow-up report from the PACE trial.

I provided a  heavily accessed blog post on follow-up study, an invited critical letter with Keith Laws to The Lancet Psychiatry, as well as a slideshow for a hastily prepared pub talk in Edinburgh that is now received over 14,000 views. I obviously don’t like the follow up study and you can look at any of these materials to see my reasons.

You can go to the Science Media Centre to see  Professor Moss-Morris’ expert opinion. She was quite embarrassed to have been brought in and declared a conflict of interest generally missing in this kind of publicity campaign:

Declared interests

Prof. Rona Moss-Morris: “Two authors of this study, Trudie Chalder and Kimberley Goldsmith, are colleagues of mine at King’s College London. I work with Trudie on other CFS work and with Kimberley on different work.  I published a small study on GET in 2005. I am a National Advisor for NHS England for improving access to psychological therapies for long-term conditions and medically unexplained symptoms. Peter White (another author of the present study) is Chair of trial steering committee for an HTA NIHR-funded RCT I am working on with people with irritable bowel syndrome.”

So, here you have me offering an evaluation of a media event that I did not actually attend. How vexatious of me, but I at least offer ample documentation of my complaints and concerns. Take issue with me in comments here if you like or you can simply swallow what the Science Media Centre feeds to you.

Institutions and websites like SMC which brand themselves as trusted sources should offer independent critical evaluations of claims that investigators are making about their favorite interventions. Otherwise, they are serving as l public relations machines.

 

What is next for Coyne of the Realm after Queen Mary University London releases PACE trial data?

I was without Internet until my plane from Zurich touched down in Philadelphia. Gmail wasn’t loading as we taxied in on the tarmac. As a result, my first notification of the Queen Mary University London (QMUL) release of the PACE chronic fatigue syndrome trial data was a direct Facebook message from an Irish student, Aine Hefferon.

Aine’s message contained a link to an excellent commentary describing the events leading up to the release. It notes:

The data was requested in March 2014 by Alem Matthees, in order to allow the calculation of the trial’s main outcomes and recovery rates according to the methods specified in the trial’s original protocol. The original analysis methods were abandoned once the trial was underway and replaced by others, including an analysis in which patients could become more disabled and yet be classed as having “recovered”.

Tom Kindlon, a patient whose criticism of PACE’s analyses has been published in medical journals, said, “This is a great day for patients. We’ve waited years for this. Finally, it’s going to be possible for independent parties to scrutinise the data and, in particular, find out what the results would have been without all the unjustified changes to the study protocol. Looking at how the objective data relate to the subjective outcomes will also be very interesting.”

He added, “This was a publicly funded trial and cost £5 million in taxpayers’ money — the data should never have been kept secret. It is very disappointing that both the PACE Trial investigators and QMUL fought the case so hard, forcing Alem Matthees to have to put in so much work when he is not well himself, and dismissing some other requests for basic information.”

The day before the data was released, the PACE authors published online the main results for the trial using the original protocol-specified methods. The new results show that only a third as many patients improved according to the protocol-defined analysis, compared to the numbers reported in The Lancet in 2011.

Because the data are being released as a result of a Freedom of Information Act request, the data will be publicly available for all to analyze. I doubt there will be any surprises for those of us who have long suspected, with good reason, that the PACE investigators based their claims of the efficacy of cognitive behavioural therapy (CBT) and graded exercise therapy (GET) on protocol violations including switching of key primary outcome variables and suppression of the secondary objective data.

I’ve noted  that this release of the data is independent of my request of data from the investigators which were promised as a condition for publishing another, cost-effectiveness analysis paper in PLOS One.  My request was not made under the Freedom of Information Act, but was construed as such by the PACE investigators. They annoyingrefused the request and labeled me as “vexatious” for having made it.

Months after posting a comment that could be construed as an Expression of Concern (but not explicitly labeled as such), PLOS One has neither obtained the data for me nor retracted the journal paper for violation of the terms under which it was published.  Downloads of the paper from the website still carry no indication of its status as questionable science.

I continue to be blocked from blogging about the situation at the PLOS blog Mind the Brain, despite my previous blog post about the PACE trial having been one of the most accessed of any thousands posted on the large family of PLOS blog posts of 2015.

As one of PLOS One’s Academic Editors,  I will continue my refusal to receive papers until I have a discussion with Iratxe Puebla, Managing Editor for PLOS ONE that is moderated by an independent third party. A fruitful discussion would involve the journal publicly acknowledging the vaguely communicated ban on my blogging about the trial that resulted in my access to the blog temporarily being blocked in December 2015 with all posting of commenting by readers being stopped as well. The discussion would also involve disclosure of the one-sided, non-transparent process that has occurred with Queen Mary University London and the PACE investigators, to the exclusion of other key stakeholders.

I’m not holding my breath until that occurs. I am announcing three steps that will take place in the near future.

  1. I will this week test the status of routine data-sharing in the United Kingdom with a formal request for data from a paper published in The Lancet. Access to the data are necessary  for independent evaluation of dubious claims being made that behavioural activation for depression delivered by minimally trained providers is suitable for replacing CBT delivered by clinical psychologists.
  1. I will provide scientifically – and ethically- based arguments why the parents in the UK should consider not enrolling their children in the MAGENTA trial and why they should withdraw their children if they have already consented to their participation, until certain conditions are met. It would be wise for the investigators conducting a trial to suspend recruitment until some key issues are considered.
  1. I will resume a dialogue  with Dr David Tovey, Editor in Chief, The Cochrane Library, and Deputy Chief Executive Officer about the need to rectify the situation in which the PACE investigators intruded upon what should have been an independent Cochrane review of graded exercise for chronic fatigue syndrome  Armed with new evidence produced in the Tribunal and subsequently, I will argue that the review should be withdrawn or have an appropriate Expression of Concern attached. Furthermore, the Cochrane should re-organize the ill formed and misnamed Common Mental Disorders Group so Myalgic Encephalomyelitis/Chronic Fatigue Syndrome is excluded. Considering ME/CFS as a “mental disorder” is out of sync with the rest of the world, where these conditions are considered physical illnesses. Putting these physical disorders under the rubric of common mental disorders is archaic and a blatant concession to vested interests in the UK.

Release the PACE trial data: My submission to the UK Tribunal

 

ben franklinThe UK Tribunal has denied the appeal of Queen Mary University London on behalf of the investigators of the PACE chronic fatigue syndrome trial and ordered the release of the trial data. This landmark decision is still subject to further appeal.

You can get further details of the decision in my recent blog posts here and here.

I was not a party to the appeal. My request for the PACE trial data was for what had been promised in the submission of a manuscript and publication as an article in PLOS One. Despite its subsequent portrayal by the PACE investigators, my request was not made under the Freedom of Information Act.

I was, however, approached for advice and asked to co-sign of a draft joint statement of support. I responded on  letterhead but on the joint statement ended up actually getting submitted.

Nonetheless,  Trudie Chalder cited in testimony before the Tribunal my blogging activity and social media as an example of the reputational threat posed to the PACE investigators if they were to release their data for re-analysis. The Information Commissioner countered that I represented the quality of academics seeking independent analysis of the PACE data. The Tribunal accepted this argument. When it becomes available, I will post this testimony.

My letter of support

penn seal

Hospital of The University of Pennsylvania

March 25, 2016

JOINT LETTER TO THE FIRST-TIER TRIBUNAL

To the members of the First-tier Tribunal (Information Rights) who are handling case EA/2015/0269:

I am writing in support of the respondents, the Information Commissioner and Mr Alem Matthees.

Researchers, academics, scientists, journalists, patients, advocates, and others have expressed numerous reasonable concerns over how the PACE trial was conducted, analyzed, and reported. Over 12,000 individuals have signed a petition calling for an independent review of the PACE trial and for the release of anonymised individual patient data from the trial.

I believe these concerns warrant the disclosure of the requested data, to enable re-analysis of the trial results in a manner which can be easily confirmed and debated by others in the spirit of open science. In accordance with the Information Commissioner’s decision, I do not believe the requested data poses a significant risk of identification to trial participants. The public interest in disclosure also outweighs the interests of QMUL.

The outcome of this case will have significant consequences for the patient community and the wider research community.  Disclosure will help greatly to address important areas of scientific debate, resolve the controversy over the trial results, alleviate the distress of the patient community, and restore public trust in UK research. Non-disclosure would simply prolong the controversy and could cause further damage to the reputation of research in the UK, and to the “open science” movement in general. I also fear that non-disclosure will set a negative precedent.

I respectfully ask the Tribunal to consider the concerns I describe above, and reject the appeal from the appellant, Queen Mary University of London (QMUL), and to uphold the previous decision made by the Information Commissioner on 27 October 2015. Alternatively, I also respectfully call upon QMUL to withdraw their appeal.

Thank you for your consideration.

Sincerely,

James C. Coyne, Ph.D.

Professor Emeritus of Psychology in Psychiatry

 

QMUL responds to UK Tribunal ordering release of PACE chronic fatigue syndrome trial data

pace trialStakeholders from around the world are reviewing the 48 page document announcing the long-awaited decision of the UK Tribunal. The decision rejects the appeal of the PACE investigators of the previous Information Commissioner’s ordering of release of the PACE data.

 A searchable, extractable PDF of the document is now available here.

I encourage you all to examine it and see how the arguments for and against data-sharing were exchanged and evaluated. Many of the arguments advanced by QMUL against routine data sharing have been previously presented by their surrogates organized by the Science Media Centre. Undoubtedly, we will see these arguments again and it’s nice to see them effectively demolished in this document. The battle for routine data-sharing is far from over.

patients say release the pace data There is much more to be said about this, but now Queen Mary University of London has issued a statement. It is reproduced below. It’s notable that it focuses on the decision having been reached in a 2:1 majority.

 The single dissenting opinion was from the lay member of the Tribunal,  who was apparently persuaded by a witness for QMUL that it just might be possible for  someone with an enormous amount of time and inordinate skills to reconstruct one or two identities of specific patients from the anonymized data.

Ross Anderson

Ross Anderson

In the majority decision, this expert witness, Ross Anderson was thoroughly demolished and criticized for his speculative arguments and conflict of interest.

The evidence of Professor Anderson that third parties could not identify participants from the information alone and that, when pressed, he said the chance of an “activist” being able to discover information that would lead to individual identification was remote. It was clear that his assessment of activist behaviour was in our view, grossly exaggerated and the only actual evidence was that an individual at a seminar had heckled Professor Chalder. The identity of those questioning the research, who signed an open letter or supported it, was impressive. While we accept that Professor Anderson was an expert witness, he was not a Tribunal appointed independent witness but appointed by the Appellant and clearly, in our view, had some self-interest, exaggerating his evidence and did not seem to us to be entirely impartial. What we got from him was a considerable amount of supposition and speculation with no actual evidence to support his assertions account the Respondents’ arguments

QMUL’s reaction:

Statement: Disclosure of PACE trial data under the Freedom of Information Act

Tuesday 16 August 2016.

As anticipated, however, QMUL is singling the likelihood that they will appeal this decision “, taking into account the interests of trial participants and the research community.”

Statement from Queen Mary University of London (QMUL):

A Tribunal has now concluded by a 2:1 majority that certain PACE trial data should be disclosed under the Freedom of Information Act.

The PACE trial was carried out according to the regulatory framework for UK clinical trials, which aims to ensure that trial participants can be confident that their information is only ever used according to their consent, and that their data is only shared under obligations of strict confidentiality.

QMUL’s appeal against the Information Commissioner argued in favour of controlled and confidential access to patient data from the PACE trial. QMUL has shared data from the PACE trial with other researchers only when there is a confidentiality agreement in place and an agreed pre-specified statistical plan for data analysis.

This has been a complex case and the Tribunal’s decision is lengthy. We are studying the decision carefully and considering our response, taking into account the interests of trial participants and the research community.

 

TRIBUNAL ORDERS RELEASE OF PACE TRIAL DATA

tribunal-de-cristo-trono-brancoAugust 16, 2016, 8:51 a.m EST: I’ve only just received a scan of the 48 page judgment. Because the scan is not searchable, I dictated the closing Majority decision.

[Update  8:00 pm: A searchable full PDF of the judgment is now available here.]

But to get to the punch line:

DECISION:

The Tribunal, by a majority, upholds the decision notice dated 27 October 2015 and dismisses the appeal.

After some brief introductory comments from the blog of Attorney Valerie Elliott Smith, I present some details from the majority decision. I will continue to review the whole document, but I think the decision represents a smashing defeat for the PACE investigators, in which their detailed arguments were turned back on them. It is also defeat for the many foes of routine data sharing, including those hastily marshaled by the Science Media Centre.

PLOS headquartersIn terms of my own quest for the PLOS One PACE data, recall that I never framed my request as falling under the Freedom of Information Act rubric. I simply asked that the data be provided to me as the PACE investigators had promised. There has been no consultation with me nor other stakeholders in the many months of the Senior Management of PLOS One’s negotiation with the PACE investigators. Maybe the Tribunal decision will now break the stalemate and encourage some transparency on the part of the senior management of PLOS One. However, I’m continuing with plans to go to Cambridge in November or early December and march on the PLOS headquarters and demand an open dialogue. I welcome you to join me and we will party outside the headquarters. Or maybe not – I hope, but I cannot expect the matter to be resolved by then.

Finally, I find particularly delicious a section of the majority decision that I reproduced below:

In any event there is a strong public interest in releasing the data given the continued academic and for so long after the research was published and seeming reluctance to Queen Mary University to engage with other academics they thought were seeking to challenge the findings (evidence of Professor Chalder).

Trudie Chalder  used the scarce time allocated to her appearance before the tribunal specifically to continue her attack on my character. She expressed alarm over the  threat my criticism posed to the reputation of the PACE investigators. Trudie, vexatious, my ass. You have been rude and unprofessional. You have consistently refused to debate, but you can look forward to a future detailed critique of your miserable, p-hacked mediational analysis.

 Attorney Valerie Elliott Smith immediately blogged with some background, to which she promises to add more later.

TRIBUNAL ORDERS RELEASE OF PACE TRIAL DATA (QMUL v the IC and Matthees)

The First-Tier Tribunal judgment in this case has just been published. QMUL’s appeal has been roundly dismissed and therefore the Tribunal has decided that the requested data from the PACE trial should be released.

I have just skimmed the 48 pages of the judgment and so have only taken in a small amount so far. However, it appears that this is a defining moment for the international ME community and the PACE Trial. Alem Matthees (the original requestor of the data) has done an extraordinary job.

However, it is important to remember that, in theory,  QMUL could still seek leave to appeal against this judgment to the Upper Tribunal so it will be a bit longer before we can be absolutely certain that this judgment will stand.

Background note for new readers

In March 2014, Mr Matthees sought some of the data from the controversial PACE trial, using the process set out in the English Freedom of Information Act (FOIA). This information is held by relevant public authority, Queen Mary University of London (QMUL). QMUL refused to disclose the data.

In due course, Mr Matthees complained to the Information Commissioner (IC) who, in October 2015ordered that the information be disclosed. QMUL appealed against the IC’s decision; that appeal was heard by the First-Tier Tribunal on 20-22 April 2016 in central London. QMUL and the IC were legally represented and QMUL called witnesses to give evidence. Mr Matthees had been joined as a party to the proceedings. He was not legally represented and did not attend the hearing but made written submissions. Judgment is awaited.

[Note: the PACE trial, which was published in 2011, relates to certain treatments for the condition known as “chronic fatigue syndrome” (CFS). CFS is often conflated (confusingly) with myalgic encephalomyelitis (ME) and referred to as CFS/ME or ME/CFS, to the detriment of genuine ME patients. This is the situation in many countries and has been for decades; it is the cause of significant confusion and distress to many patients worldwide.

The results of the PACE trial appear to promote psychosocial treatments which many patients find either ineffective or actively harmful. As a result, some patients have been using FOIA to try to obtain the trial data in order to understand how these results were achieved. However, most requests have been denied and, five years on, most of the data is still unavailable.] 

The Tribunal decision [Please alert me to typos in this section that was hastily entred with vocie recognition software.]

DECISION:

The Tribunal, by a majority, upholds the decision notice dated 27 October 2015 and dismisses the appeal.

The majority decision

The Commissioner accepts that the question turns on whether anonymization is possible, and he argues that in this instance identification is an extremely remote possibility. Professor Anderson accepted that the information alone cannot identify participants, and his hypothesis that identification is possible to combine that information with NHS data (involving an NHS employee both having breachedtheir professional, legal and ethical obligations and also having the skill and inclination to do this) is implausible.

In short we accept and adopt the Commission’s wider submissions and reasoning as set out in his Skeleton Arguments and Written Closing on this issue. In all circumstances and on the evidence before us we are satisfied that the risk of a dedication has been anonymized to the extent that the risk of identification is remote. Incomes this conclusion we’ve also taken into account:

  1. The nature of the information, which did not contain any fixed or direct identifiers;
  2. The evidence of Dr. Rawle that the anonymization methodology followed the guidelines at the time and would still comply with current guidelines although they were said to be under review for the future;
  3. The evidence of Dr. Rawle that none of the identifiers were contained in the disputed information, (the anthropometry measurement issues were cleared up);
  4. The evidence of Professor Anderson that third parties could not identify participants from the information alone and that, when pressed, he said the chance of an “activist” being able to discover information that would lead to individual identification was remote. It was clear that his assessment of activist behavior was in our view, grossly exaggerated and the only actual evidence was there an individual at seminar had heckled Professor Chalder. The identity of those questioning the research, who signed an open letter or supported it, was impressive. While we accept that Professor Anderson was an expert witness, he was not a Tribunal appointed independent witness but appointed by the Appellant and clearly, in our view, had some self-interest, exaggerating his evidence and did not seem to us to be entirely impartial. What we got from him was a considerable amount of supposition and speculation with no actual evidence to support his assertions account the Respondents’ arguments;
  5. That even on Professor Anderson’s evidence for identification to take place there would have to be a breach of medical ethics and law and there was actually no evidence to quantify the risk of this occurring in the circumstances and on the facts before us in the appeal. In fact there was no tangible evidence of an example where such steps had led to a identification of the individual in any circumstances.
  6. That even on Professor Anderson’s it was only the walking scores that were likely to lead to identification (if all of his other suppositions and speculations came about);
  7. That the Fine research had been released, albeit t by accident, and there was no evidence that although it contains similar data, that there had been any individual identification or problems arising from it.
  8. We do not accept the commercial interest arguments of the College. There was very little evidence of the withdrawal of consent and where it happened it was only directly related to the issues before us. Funding had been obtained for new trial in the knowledge that the tribunal may not allow this Appeal. New confidentiality guarantees, or perhaps more explicit ones, could be given to new participants.
  9. In any event there is a strong public interest in releasing the data given the continued academic interest so long after the research was published and seeming reluctance to Queen Mary University to engage with other academics they thought were seeking to challenge the findings (evidence of Professor Chalder).
  10. There is insufficient evidence before to persuade us that the disclosure of the disputed information would cause sufficient prejudice to QMUL’s research programmes, reputation and funding streams.
  11. Professor Anderson when cross examined as to whether or not the patient identities (HESID) that would disclose to anyone with access were encrypted. He was unaware but has since checked and is now confirmed through submissions on behalf of the Appellant that this does not affect the wider point he made about care data disclosures, that they would potentially allow different health events to be linked where it could be established that they relate to the same patient identifier. This, we would have preferred to explore further in evidence but it seems to us that encryption makes the chance of identification even more remote in any event and strengthens our view that the speculative assertions of the occurrence of possible events actually taking place in a way that could lead to identification individuals, by Professor Anderson are indeed remote.
  12. We do not accept the speculation that the chanmce of a determined person with specialist skills could make the link, while less than probable, is more than remote. There’s no tangible evidence before us to persuade us that is less than remote. Professor Anderson accepted it would be extremely difficult to identify individuals even from the collective information, which as the Commissioner submits “given his approach to this would indicate the near impossibility of reidentification”. We are not persuade the risk of identification is more than remote.
  13. Generally, regarding the Commissioner’s discretion, we heard nothing, and are not persuaded on the evidence before us, that would lead us to question that the Commissioner had not apply himself correctly ,  that his decision was not properly arrived at or should be set aside.

[The majority Decision – (Mister Watson dissenting);

  1. Each role in the spreadsheet is unique and refers to one person in the trial.
  2. The information necessary to link the data to an individual is available to a large number of people due to the way in which security has been implemented and the NHS and the quantity and nature of the information is now available on social media.
  3. I believe Professor Anderson is correct when he gave evidence that the chance they determined person with specialist skills could make the link, while less than probable, is more than remote.
  4. For this reason the information contained in the spreadsheet is personal data and should not be disclosed.

Finally, at paragraph 5 c) above: “Would disclosure cause sufficient prejudice to QMUL’s research programmes, reputation and funding streams to refuse disclosure?”

We unanimously accept and adopt the Commissioner’s wider submissions and reasoning has set out in his Skeleton Arguments and Written Closing on this issue. In all the circumstances and on the evidence before us we are satisfied that the perceived risk of prejudice as submitted by the Appellant’s has not been substantiated or demonstrated evidence before us. Such minimum risk as has been expressed would not now view outweigh the public interest in disclosure of the disputed information as defined in the specific request in this appeal.

  1. The Tribunal wish to thank all parties for the helpful manner in which they have presented their arguments and submissions. We’ve been provided with an extraordinary amount of anciliary and background information on and about the important subject matter under consideration and have considered all of it. They can be no doubt about the Public Interest in the subject matter which is evident through the course of this appeal, and beyond, and we are grateful for the assistance has been given to us in this regard.

We have considered all of the above arguments, submissions and evidence together with a significant volume of supporting evidence in legal precedents and for the reasons given above we refuse the appeal by majority decision for the above reasons, and the Commissioner’s DN stands.

Brian QC 11th August 2016

Promulgated 12th  August 2016

 

 

CBT versus psychodynamic therapy for depression: One sentence changes the whole story

A recent comparative effectiveness study in JAMA Psychiatry of CBT versus psychodynamic psychotherapy for depression was billed as a noninferiority trial.

booby prizeOne sentence in the results section changed the whole significance of the study.

The dodo bird verdict for the study is that everybody gets a booby prize.

The study is currently freely accessed at JAMA Psychiatry, although you may need to register for free to actually download the PDF.

 

Connolly Gibbons M, Gallop R, Thompson D, et al. Comparative Effectiveness of Cognitive Therapy and Dynamic Psychotherapy for Major Depressive Disorder in a Community Mental Health Setting: A Randomized Clinical Noninferiority Trial. JAMA Psychiatry. Published online August 03, 2016. doi:10.1001/jamapsychiatry.2016.1720.

The moderately sized study compared to active treatments without a nonspecific comparison/control group.

Results.  Among the 237 patients (59 men [24.9%]; 178 women [75.1%]; mean [SD] age, 36.2 [12.1] years) treated by 20 therapists (19 women and 1 man; mean [SD] age, 40.0 [14.6] years), 118 were randomized to DT and 119 to CT. A mean (SD) difference between treatments was found in the change on the Hamilton Rating Scale for Depression of 0.86 (7.73) scale points (95% CI, −0.70 to 2.42; Cohen d, 0.11), indicating that DT was statistically not inferior to CT. A statistically significant main effect was found for time (F1,198 = 75.92; P  = .001). No statistically significant differences were found between treatments on patient ratings of treatment credibility. Dynamic psychotherapy and CT were discriminated from each other on competence in supportive techniques (t120 = 2.48; P = .02), competence in expressive techniques (t120 = 4.78; P = .001), adherence to CT techniques (t115 = −7.07; P = .001), and competence in CT (t115 = −7.07; P = .001).

Conclusions and Relevance.  This study suggests that DT is not inferior to CT on change in depression for the treatment of MDD in a community mental health setting. The 95% CI suggests that the effects of DT are equivalent to those of CT.

In case there is any ambiguity in the message the authors wanted to convey, they reiterated:

Key Points

  • Question Is short-term dynamic psychotherapy not inferior to cognitive therapy in the treatment of major depressive disorder (MDD) in the community mental health setting?

  • Findings In this randomized noninferiority trial that included 237 adults, short-term dynamic psychotherapy was statistically significantly noninferior to cognitive therapy in decreasing depressive symptoms among patients receiving services for MDD in the community mental health setting.

  • Meaning Short-term dynamic psychotherapy and cognitive therapy may be effective in treating MDD in the community.

Despite an accompanying editorial, the study only got a moderate amount of immediate attention in the social media. Here are the altmetrics.views

altmetrics PNG

parroting I examined the 40 tweets available on August 6, 2016 and found only one that went beyond parroting.

good tweets I I suspect that Robert Howard had discovered the one sentence in the results section that I noticed:

 

Nineteen patients (16.1%) in DT and 26 patients (21.8%) in the CT condition demonstrated response to treatment as measured by a 50% reduction on the HAM-D score across treatment (χ21 = 1.27; P = .32).

Most of the patients assigned to either group in this study failed to respond to treatment. Tipped off by this sentence, I looked for the degree of treatment exposure and found that most patients did not get exposed to sufficient intensity of treatment.

Sixty-three patients (26.6%) attended 1 or fewer sessions of psychotherapy; 122 (51.5%), 5 or fewer sessions; and 187 (78.9%), 11 or fewer sessions. We found no statistically significant difference between treatments in the number of sessions attended (t235 = 1.47; P = .14).

 The title of the JAMA Psychiatry article noted that patients had been recruited from the community mental health center. I interpret this to suggest they were likely to be a low income group who were not previously prepared for psychotherapy.

Before anyone proposes that the solution is simply to offer more therapy, note that the patients were not attending enough sessions of a larger number (16) that were offered. My interpretation is that greater effort may be needed to get such patients to consistently show up for sessions.

My colleagues and I previously conducted an exceptionally well resourced study in in the same low income and socially disadvantaged Philadelphia population. Our intention was to reduce risk factors among recently pregnant, low income women for another low weight birth delivery. We demonstrated that we could recruit and retain these women, but it took an intensive, creative effort.

One of the risk factors that we addressed was depression and we offered antidepressant medication and free treatment at the world-renowned University of Pennsylvania Center For Cognitive Therapy. We provided free transportation and child care. Few women access sufficient therapy or receive sufficient dose of antidepressants. The therapists at the center complained that the women did not seem to have their life in order and did not seem ready for psychotherapy. Personally, I think that the therapist may not have been ready for such women and did not sufficiently engage them.

Back to the study under discussion, it was accompanied by an editorial that parroted the authors’ intended message in its title:

Abbass AA, Town JM. Bona Fide Psychotherapy Models Are Equally Effective for Major Depressive Disorder: Future Research Directions. JAMA Psychiatry. Published online August 03, 2016. doi:10.1001/jamapsychiatry.2016.1916.

But I noticed this in the text:

EFFECTIVENESS OF PSYCHOTHERAPY FOR MDD: GOOD BUT COULD BE BETTER?

Among other points, the study by Connolly Gibbons and colleagues raises the ongoing challenge facing all psychiatrists using pharmacotherapy and psychotherapy: how to improve rates of remission in real-world clinical samples. The study found that more than 80% of all participants did not respond to treatment (22% of patients receiving CBT and 16% of patients receiving STPP had response to treatment as measured by a 50% reduction in observer-rated depression). This high rate of nonresponse may be partly explained by inadequate treatment “dose” or number of sessions, clinical sample, therapist expertise, biomedical factors, and sociofamilial factors impeding outcomes

The JAMA Psychiatry article under discussion cited another, similar study conducted in the Netherlands, but did not elaborate on its findings:

Driessen E, Van HL, Don FJ, Peen J, Kool S, Westra D, Hendriksen M, Schoevers RA, Cuijpers P, Twisk JW, Dekker JJ. The efficacy of cognitive-behavioral therapy and psychodynamic therapy in the outpatient treatment of major depression: a randomized clinical trial. American Journal of Psychiatry. 2013 Sep 1.

Unlike the JAMA Psychiatry article, the abstract of the Dutch study qualified its finding of non-inferiority by noting that nether therapy did particularly well:

RESULTS:

No statistically significant treatment differences were found for any of the outcome measures. The average posttreatment remission rate was 22.7%. Noninferiority was shown for posttreatment HAM-D and patient-rated depression scores but could not be demonstrated for posttreatment remission rates or any of the follow-up measures.

CONCLUSIONS:

The findings extend the evidence base of psychodynamic therapy for depression but also indicate that time-limited treatment is insufficient for a substantial number of patients encountered in psychiatric outpatient clinics.

dodo bird verdictI suspect that both of these randomized trials will be cited as evidence of the Dodo Bird Verdict for psychotherapy for depression – everybody’s a winner and everybody gets a prize. However, in both the studies, the cognitive behavior therapy underperformed relative to the efficacy demonstrated in a larger body of studies. The literature for psychodynamic therapy is more limited and of low quality.

Still, I think the messages that when you move into more difficult populations, you can’t expect results obtained with more carefully selected, therapy-ready patient populations who were recruited to more typical studies. But this may reflect on the unrepresentativeness of patients in the larger literature.

Meanwhile, Psychiatrist Erick Turner and I have been having an exchange on Twitter concerning another noninferiority study.

Turner Tweet.PNG

Erick is referring to a perspective he shares with things I’ve been saying regularly about noninferiority trials. They typically don’t include a nonspecific comparison/control group. Without such a group, we can’t evaluate whether either of the active treatments are better than provision of nonspecific treatments with elements of support, positive expectation, and attention.

That is also a limitation of the current study, but by peeking into the actual results, we discover referral to neither of two active treatments left most patients free of depression.

What if there had been a credible attention/support condition in the present study? Would either of these two treatments that were “noninferior” to each other have a clinically significant advantage? What would be the implications, if not? would the report have made it into JAMA Psychiatry?

Too much ado about church attendance and suicide rates among women

 VanderWeele TJ, Li S, Tsai AC, Kawachi I. Association Between Religious Service Attendance and Lower Suicide Rates Among US Women. JAMA Psychiatry. 2016;73(8):845-851. doi:10.1001/jamapsychiatry.2016.1243.

praying inThis recent study in JAMA Psychiatry received an extraordinary amount of attention in a short time, undoubtedly orchestrated by the journal. Prestigious journals interested in keeping their prestige precisely track how much immediate attention papers get using altmetrics.

The intent is to boost immediate attention, which increases early citations. In turn, early citations raise impact factors. Journal impact factors are calculated based on the number of citations the papers received two years after publication. Journals can advertise higher impact factors which boost subscriptions and paid advertisements. Here are the altmetrics for this article. They suggest it is an outlier in terms of the amount of attention it got in a short time.

views

shares

Among the numerous sources of attention were religious oriented media. Catholic Online proclaimed How the Church can improve your life. But secular medical sources simply proclaimed Take Me To Church: Attending Religous Services Linked To Lower Suicide Rates Among Women.

An article in the UK Spectator, People who go to church live longer. Here’s why was written by the first author of the JAMA Psychiatry paper, but I had to compare authorship of the articles to discover this.

During such publicity campaigns, journals often temporarily provide free access to what would otherwise be pay walled articles in order to stimulate attention. Unfortunately, that wasn’t the case with this article, and so readers without access to a subscription or a university library could only check the claims against an abstract, not the full paper. Here’s an excerpt:

Design, Setting, and Participants  We evaluated associations between religious service attendance and suicide from 1996 through June 2010 in a large, long-term prospective cohort, the Nurses’ Health Study, in an analysis that included 89 708 women. Religious service attendance was self-reported in 1992 and 1996. Data analysis was conducted from 1996 through 2010.

Results. Among 89 708 women aged 30 to 55 years who participated in the Nurses’ Health Study, attendance at religious services once per week or more was associated with an approximately 5-fold lower rate of suicide compared with never attending religious services (hazard ratio, 0.16; 95% CI, 0.06-0.46). Service attendance once or more per week vs less frequent attendance was associated with a hazard ratio of 0.05 (95% CI, 0.006-0.48) for Catholics but only 0.34 (95% CI, 0.10-1.10) for Protestants (P = .05 for heterogeneity). Results were robust in sensitivity analysis and to exclusions of persons who were previously depressed or had a history of cancer or cardiovascular disease. There was evidence that social integration, depressive symptoms, and alcohol consumption partially mediated the association among those occasionally attending services, but not for those attending frequently.

An accompanying editorial

Koenig HG. Association of religious involvement and suicide. JAMA Psychiatry. 2016 Jun 29.

had free access.  It extolled the virtues of the study:

The study by VanderWeele et al is important because of the large sample, lengthy follow-up period, and rigorous statistical methods used to analyze the data, including adjustments for baseline religious service attendance and removal of women who were previously depressed or with major physical health problems. Adjustment for baseline religious attendance and initial removal of women with depression or physical illness is particularly important to avoid the problem of reverse causation, an issue in studies of religious service attendance and other health outcomes. Depressed persons at greatest risk for suicide are often socially withdrawn and less likely to attend religious services, which could otherwise explain the association.

The editorial suggested immediate clinical applications of the findings, but ended on a discreet note of caution that would set off alarm bells for a skeptic.

 What should mental health professionals do with this information? Evaluating patients’ moral beliefs about suicide and level of involvement in religious community may help clinicians gauge risk of suicide. Thus, the findings by VanderWeele et al underscore the importance of obtaining a spiritual history as part of the overall psychiatric evaluation, which may identify patients who at one time were active in a faith community but have stopped for various reasons. Exploring what those reasons were, particularly among the socially isolated, and perhaps supporting a return to such activity, if the patient desires, may help produce social connections that lower suicide risk. If based exclusively on the findings of VanderWeele et al, these suggestions would apply only to white female nurses in the United States. Given the large amount of research in other ethnic, sex, socioeconomic, professional, and nonprofessional groups that shows similar associations, one might be tempted to apply these findings to other populations as well. Nevertheless, until others have replicated the findings reported here in studies with higher event rates (ie, greater than 36 suicides), it would be wise to proceed cautiously and sensitively.

This last sentence provides crucial information that should have been reported in the abstract of the article. The seemingly impressive study involved predicting only 36 suicides. Any multivariate analyses spread these 36 suicides across little boxes of categorical variables. Imprecision in the measurement of any of these variables or any misclassification could produce very different results.

As a skeptic accustomed to hard-sell efforts based on weak data, this is the information I would have immediately sought. Armed with it, I would’ve been prepared to reject as nonsensical the pseudo-precision of a dramatic estimates of effacts that are contained in the abstract..

 …Attendance at religious services once per week or more was associated with an approximately 5-fold lower rate of suicide compared with never attending religious services (hazard ratio, 0.16; 95% CI, 0.06-0.46).

Come on! This “five-fold” difference refers to differences in the distribution of fewer than 36 suicides in a couple of boxes. It would have been less striking if it were reported in absolute terms. The boxes are “no attendance,” versus “attendance once a week or more.” That means that some of the already small number of 36 suicides were thrown out because they did not occur among women who attended church but less consistently than once a week or more.

Dropping subjects in the middle of a distribution and focusing on extremes inflates the significance of results. There is no evidence that these authors made the decision to do so without first peeking at their data.

Seemingly impressive multivariate analyses that build on these analyses just add further nonsense and noise.

Women who have the regularity of routine to show up in church at least once a week or more are often regular in other ways they can affect the health and well-being.

The first is called “healthy user bias.” As Gary Taubes described nicely, “people who faithfully engage in activities that are good for them — taking a drug as prescribed, for instance, or eating what they believe is a healthy diet — are fundamentally different from those who don’t.

Women who regularly go to church take better care of themselves in still other ways.

 Next, there is another subtle component of healthy-user (or “healthy continuer”) bias. This is the “compliance or adherer effect or bias”. Individuals who comply or adhere with their doctors’ orders when given a prescription are different and healthier than people who don’t.

This Nurses Health cohort is well studied. In lots of papers. Its many biases  have been pointed out. To start with, it’s limited to nurses. Extremes of social deprivation are excluded. And the rates of suicide in the Nurses Health Study are substantially less than in the general population of women

I don’t have much confidence in the claims being made in the press coverage or the article itself.But my skepticism turned to disappointment in the authors. I noticed that they did that did not adequately acknowledge results concerning suicide from the same Nurses Health Study set that were published less than a year ago in the same journal.

Tsai AC, Lucas M, Kawachi I. Association between social integration and suicide among women in the United States. JAMA Psychiatry. 2015 Oct 1;72(10):987-93.

Unlike the present study, this one is available as a PDF without a pay wall. For reasons that are explained, it has 43 deaths by suicide to explain and suggests that social integration is a reasonably good predictor of death by suicide. This is a contradiction of what is said in the current paper.

Who knows why. But dropping a few death by suicide can make a substantial difference when there are so few to begin with.  Then there is always the voodoo of applying multivariate statistics in predicting so few infrequent events. I think this further demonstrates the problems of making a big fuss when talking about so few suicides. Add or subtract only a few suicides and  you get a whole different story to tell.

I hope you have  learned some things from this exercise:

  1. Whenever you see reports of epidemiological studies of suicide, keep in mind the infrequency of suicide. Pay attention to the number of death by suicide to explain, not the size of the overall sample.
  2. Correlation does not equal causality, particularly when the correlations occur with multivariate statistical analyses with unknown peeking at the data ahead of time.
  3. Beware of clinical implications being drawn from such weak data.