Systematic review shows no improvement in quality of mindfulness research in 16 years

Should we still take claims about mental health benefits of mindfulness with a grain of  salt? A systematic review by one of mindfulness training’s key promoters suggests maybe so.

saltCritics have been identifying the same weaknesses in mindfulness research for almost two decades. This review suggests little improvement in 16 years the quality of randomized trials for mental health problems.

This study examined 171 articles reporting RCTs for:

(a) active control conditions, (b) larger sample sizes, (c) longer follow-up assessment, (d) treatment fidelity assessment, (e) reporting of instructor training, (f) reporting of ITT samples.

What was missed

Whether articles reporting RCTs had appropriate disclosure of financial or other conflicts of interest. COI pose significant risk of bias, especially when they are not reported.

This article discloses authors’ interests. One of the authors, Richard Davidson is a prominent promoter of mindfulness training.  A Web of Science search of Davidson RJ and mindfulness yielded 26 articles from 2002 to 2016. It would be interesting to check in see if these consistent weaknesses in mindfulness research are mentioned in these articles. To what extent do RCTs with Davidson as an author had these weaknesses, like being underpowered?

Critic: You say financial interests or other investments in a treatment are a risk of bias. Yet, this article is critical of mindfulness research. Wouldn’t you expect a more positive appraisal of the literature because of the authors having a confirmation bias?

Not necessarily. Conflicts of interest are a risk of bias, but don’t discredit an author, They only alert readers to be skeptical. Furthermore, the weaknesses in this literature are so pervasive, it would be difficult to put a positive spin on them.  Besides calling attention to specific weaknesses that need to be addressed in future research can become part of a pitch for more research.

The article

Goldberg SB, Tucker RP, Greene PA, Simpson TL, Kearney DJ, Davidson RJ. Is mindfulness research methodology improving over time? A systematic review. PLOS One. 2017 Oct 31;12(10):e0187298.

End of paper conclusion:

In conclusion, the 16 years of mindfulness research reviewed here provided modest evidence that the quality of research is improving over time. There may be various explanations for this (e.g., an increasing number of novel mindfulness-based interventions being first tested in less rigorous designs; the undue influence of early, high-quality studies). However, it is our hope that demonstrating this fact empirically will encourage future researchers to work towards the recommendations here and ultimately towards a clearer and scientifically-informed understanding of the potential and limitations of these treatments.

From the abstract

Objectives

The current systematic review examined the extent to which mindfulness research demonstrated increased rigor over the past 16 years regarding six methodological features that have been highlighted as areas for improvement. These feature included using active control conditions, larger sample sizes, longer follow-up assessment, treatment fidelity assessment, and reporting of instructor training and intent-to-treat (ITT) analyses.

Data sources

We searched PubMed, PsychInfo, Scopus, and Web of Science in addition to a publically available repository of mindfulness studies.

Study eligibility criteria

Randomized clinical trials of mindfulness-based interventions for samples with a clinical disorder or elevated symptoms of a clinical disorder listed on the American Psychological Association’s list of disorders with recognized evidence-based treatment.

Study appraisal and synthesis methods

Independent raters screened 9,067 titles and abstracts, with 303 full text reviews. Of these, 171 were included, representing 142 non-overlapping samples.

Results

Across the 142 studies published between 2000 and 2016, there was no evidence for increases in any study quality indicator, although changes were generally in the direction of improved quality. When restricting the sample to those conducted in Europe and North America (continents with the longest history of scientific research in this area), an increase in reporting of ITT analyses was found. When excluding an early, high-quality study, improvements were seen in sample size, treatment fidelity assessment, and reporting of ITT analyses.

Conclusions and implications of key findings

Taken together, the findings suggest modest adoption of the recommendations for methodological improvement voiced repeatedly in the literature. Possible explanations for this and implications for interpreting this body of research and conducting future studies are discussed.

Competing interests

RD is the founder, president, and serves on the board of directors for the non-profit organization, Healthy Minds Innovations, Inc. In addition, RD serves on the board of directors for the Mind and Life Institute. This does not alter our adherence to PLOS ONE policies on sharing data and materials

The variables examined in the systematic review

Six methodological features that have been recommended in criticisms of mindfulness research [10–12. 14]. These include: (a) active control conditions, (b) larger sample sizes, (c) longer follow-up assessment, (d) treatment fidelity assessment, (e) reporting of instructor training, (f) reporting of ITT samples.

…We graded the strength of the control condition on a five-tier system. We defined specific active control conditions as comparison groups that were intended to be therapeutic [17]. More rigorous control groups are important as they can provide a test of the unique or added benefit a mindfulness intervention may offer, beyond non-specific benefits associated with the placebo effect, researcher attention, or demand characteristics [11,14]. Larger sample sizes are important as they increase the reliability of reported effects and increase statistical power [11]. Longer follow-up is important for assessing the degree to which treatment effects are maintained beyond the completion of the intervention [10]. Treatment fidelity assessment allows an examination of the degree to which the given treatment was delivered as intended [12]. Treatment fidelity is commonly assessed through video or audio recordings of sessions that are coded and/or reviewed by treatment experts [18]. We coded all references to treatment fidelity assessment (e.g., sessions were recorded and reviewed, a checklist measuring adherence to specific treatment elements was completed). Relatedly, reporting of instructor training increases the likelihood that the treatment that was delivered by qualified individuals [12], which should, in theory, influence the quality of the treatment provided. Lastly, the reporting of ITT analyses involves including individuals who may have dropped out of the study and/or did not complete their assigned intervention [12]. Generally speaking, ITT analyses are viewed to be more conservative estimates of treatment effects [19,20], and are preferred for this reason.

 

Unfunny 2017 BMJ Christmas articles not in the same league with my all-time favorite

Which is: A systematic review of parachute use to prevent death and major trauma

I agree with Sharen Begley’s assessment in Stat of this year’s BMJ Christmas issue as a loser.

tenorA BMJ Christmas issue filled with wine glasses, sex, and back pain brings out the Grinch in us

Bah! … humbug. Is it just us, or is the highly anticipated Christmas issue of the BMJ (formerly the British Medical Journal) delivering more lumps of coal and fewer tinselly baubles lately?

Maybe it’s Noel nostalgia, but we find ourselves reminiscing about BMJ offerings from Yuletides past, which brought us studies reporting that 0.5 percent of U.S. births are to (self-reported) virgins, determining how long a box of chocolates lasts on a hospital ward, or investigating Nintendo injuries.

I agree with her stern:

Note to BMJ editors: Fatal motorcycle crashes, old people falling, and joint pain — three of this year’s Christmas issue studies — do not qualify as “lighthearted.”

My all-time favorite BMJ Christmas article

Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials

 

With its conclusions:

As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.

The brief article is actually a great way to start a serious discussion of randomized trials with a laugh.

Parachutist-splashWe can all agree that we wouldn’t participate in a randomized trial of parachutes. Any effort to conduct a systematic review and meta-analysis of such studies would answer up , formally speaking, as a failed meta-analysis. We we could start with a rigorous systematic search, but still end up with no studies to provide effect sizes. That’s not a bad thing, especially one as an alternative to making recommendations on weak or nonexistent data.

I would like to see a lot more formally declared failed meta-analyses by the Cochrane collaboration.  Clearly labeled failed meta-analyses much preferable to recommendations consumers and policymakers for treatments based on a small collection of methodologically weak and underpowered trials. It happens just too much.

If the discussion group were ripe for it, we could delve into when randomized trials are not needed or what to do when there aren’t randomized trials. I think one or more N = 1 trials not using a parachute or similar device would be compelling, without a nonspecific control group. On the other hand, many interventions that have been justified only by observational trials turn out not to be effective when RCT is finally done.

Keep a discussion going long enough of when RCTs can’t provide suitable evidence, and you end up in a predictable place. Someone will offer a critique of RCTs as the gold standard for evaluating interventions or maybe of systematic reviews meta-analyses of RCTs being a platinum standard. That can be very fruitful too, but sooner or later can get someone proposing alternatives to the RCT because their pet interventions don’t measure up in RCTs. Ah, yes, RCTs can capture the the magic going on long-term psychodynamic psychotherapy.

[For review of alternatives to RCTs that I co-authored, see Research to improve the quality of care for depression: alternatives to the simple randomized clinical trial ]

Ghosts of Christmases Past

Searching for past BMJ Christmas articles can be tedious. If someone can suggest an efficient search term, let me know. Fortunately the BMJ last year offered a review of all-time highlights

Christmas crackers: highlights from past years of The BMJ’s seasonal issue

BMJ 2016; 355 doi: https://doi.org/10.1136/bmj.i6679 (Published 15 December 2016)

Cite this as: BMJ 2016;355:i6679

For more than 30 years the festive issue of the journal has answered quirky research questions, waxed philosophical, and given us a good dose of humour and entertainment along the way.

A recent count found more than 1000 articles in The BMJ’s Christmas back catalogue. A look through these shows some common themes returning year after year. Professional concerns crop up often, and we seem to be endlessly fascinated by the differences between medical specialties. Past studies have looked at how specialties vary by the cars they drive,7 their ability to predict the future,8 and their coffee buying habits.9 Sometimes the research findings can challenge popular stereotypes. How many people, “orthopods” included, could have predicted that anaesthetists, with their regular diet of Sudoku and crosswords, would fare worse than orthopaedic surgeons in an intelligence test?1

It notes some of the recurring themes.

Beyond medical and academic matters, enduring Christmas themes also reflect the universal big issues that preoccupy us all: food, drink, religion, death, love, and sex.

This broad theme encompasses one of the most widely accessed BMJ Christmas articles of all times

In 2014 Ben Lendrem and colleagues explored differences between the sexes in idiotic risk taking behaviour, by studying past winners of the Darwin Awards.4 As the paper describes: winners of these awards must die in such an idiotic manner that “their action ensures the long-term survival of the species, by selectively allowing one less idiot to survive.”

There is also an interesting table of the four BMJ Christmas papers that won Ig Noble prizes:

Christmas BMJ papers awarded the Ig Nobel prize

  • Effect of ale, garlic, and soured cream on the appetite of leeches (winner 1994)15

  • Magnetic resonance imaging of male and female genitals during coitus and female sexual arousal (1999)13

  • Sword swallowing and its side effects (2006)16

  • Pain over speed bumps in diagnosis of acute appendicitis (2012)17

 

 

 

Is acceptance and commitment therapy (ACT) in a post-evidence phase?

Steve Hayes encouraged me to give the evidence for the efficacy of ACT another look. I did and I wasn’t impressed.

Recent interviews with three founders of ACT make little reference to evidence, but a lot of reaching for roles as gurus, not as trainers in evidence-based therapy.

I recently tweeted about acceptance and commitment therapy (ACT).

my tweet

I got a quick reply from Steve Hayes:

edited steve hayes exchange

I checked the document, and had lots of numbers, but little indication of accumulating quality studies. I recognized some of the meta-analyses were quite bad.

ACBS

The document listed 2 organizations that describe ACT, or areas of ACT, as evidence based, APA Division 12 and SAMHA. Both sourceds depend heavily on promoters of a particular approach assembling and evaluating material, with all the biases that that introduces, not independent, systematic evaluation. The APA page was written by an ACT his enthusiasts, but was nonetheless not particularly impressive. Only ACT for chronic pain received an evaluation of more than modest support.

American Psychological Association, Society of Clinical Psychology (Div. 12), Research Supported Psychological Treatments.

Chronic Pain – Strong Research Support

Depression – Modest Research Support

Mixed anxiety – Modest Research Support

Obsessive-Compulsive Disorder – Modest Research Support

Psychosis – Modest Research Support

I checked the most recent review and meta analysis of ACT for chronic pain.

Veehof MM, Trompetter HR, Bohlmeijer ET, Schreurs KM. Acceptance-and mindfulness-based interventions for the treatment of chronic pain: a meta-analytic review. Cognitive Behaviour Therapy. 2016 Jan 2;45(1):5-31.

The review combined  28 studies evaluating mindfulness or ACT. It did not provide separate evaluations for the two treatments. There were 9 studies comparing ACT to routine care or wait list. Only one had more than 35 patients in the smallest group at follow up, a study of ACT with fibromyalgia patients. There was one study comparing  ACT with CBT having such a minimal sample size, and the two treatments did not differ. There were two comparisons with multidisciplinary treatment, but neither met this minimal sample size.

A systematic review that is almost as recent further showed what bad shape the literature concerning ACT for pain is in.

Hann, K. E. J. & McCracken, L. M. (2014). A systematic review of randomized controlled trials of Acceptance and Commitment Therapy for adults with chronic pain: Outcome domains, design quality, and efficacy. Journal of Contextual Behavioral Science, 3, 217-227.

The review discretely suggests that the weak designs and potential for p-hacking of the typical study of ACT for pain would allow even homeopathy to look better than nothing.

A systematic search identified 1034 articles and ten studies were selected as eligible for review. Overall, 15 outcome domains were assessed using 39 different measurement tools across the ten RCTs. The outcome domains assessed in the reviewed trials were, to an extent, in-line with recognized guidelines. Six of the ten studies identified primary and secondary outcomes; one included just one outcome and three did not categorize outcomes. All ten trials included a measure of some aspect of psychological flexibility; however these were not always formally identified as process variables. Pain and emotional functioning were the most frequently measured outcome domains. A review of outcome results suggests that ACT is efficacious particularly for enhancing general, mostly physical functioning, and for decreasing distress, in comparison to inactive treatment comparisons. It is recommended that future RCTs (a) formally define outcomes as primary, secondary and process variables, (b) consider including measures of physical or social functioning, rather than pain and emotional functioning, as primary outcomes, (c) address existing risks of bias, such as reporting bias, and (d) include more components of psychological flexibility, such as cognitive defusion and self-related variables.

Are the leaders of the ACT movement giving up on accumulating evidence and reaching for guru status? Maybe, if some recent interviews are any indication.

3 Founders of ACT in BPS Psychologist

Aseries of interviews by Kal Kseib with three founders of ACT can be found in the BPS Psychologist– Kelly Wilson, Kirk Strosahl and Steve Hayes.

Kelly Wilson

The title of the interview with Kelly Wilson suggests he is going to pluck some flowers stuck between his toes in his Gucchi sandals 

Kelly you find some connection

Kelly doesn’t disappoint.  He does mention evidence three times. Twice here:

There’s interesting evidence out there about how even just small doses of such things can make a difference. In research looking at sedentary behaviour, just a couple of minutes each hour of getting up and moving around is shown to disrupt some of those destructive metabolic processes. Reducing exposure to toxins including social toxins, moving your body, getting enough sleep, eating nutritious food, engaging in some small mindfulness practice – every bit of evidence seems to suggest that really small amounts matter. And if you look at behavioural activation, which is one of the tier 1 treatments for depression, it’s about ‘get them started’. The smallest thing – even if just your feet move! Go down to the corner, go out for coffee, go out to a movie with a friend. That work was really built on the idea that these patterns of small activities give a chance for the stream of life to kind of pull you back in.

And then

For instance, there’s evidence to show that people with high levels of self-stigma about their obesity massively increase their risk of all-cause mortality and cardiac mortality.

I wouldn’t interpret modest effect sizes in observational studies in this way. But whatever….

Then there is the too-much-information, grab-your-guru-status disclosure.

What’s something people might not know about you that, if they knew, would surprise them?

I came to psychology late. I was 30-years-old when I started college. I dropped out of school when I was 16. And between the ages of 16 and the age of 30, I was a drug addict and an alcoholic. Chronically unemployed and unemployable – a serial felon. Terribly, terribly depressed – suicidally depressed pretty much all the time, except for little glorious windows in time when I’d get just the right combination of drugs and I’d be able to feel like I could stay in my own skin for a minute. But mostly it was just hard and destructive to me and to a lot of other people. In 1985, at the age of 30, after many years of overdoses and car accidents and violence, illness, and just wanting to die, I was admitted into a locked psychiatric ward. I joke with people sometimes – although it’s true – that I got my start in psychology in a psychiatric hospital

I have written about aspiring TED talk gurus like Amy Cuddy and Kelly having to find their redemptive selves, linking the product they sell to overcoming the adversity in their lives. I think there may be some literary license in play here, but if it is meant to inform, rather than just entertain, what is the takeaway message?

What is the greatest lesson life has taught you recently?

That the things you love the most are also the things that are the sources of the most extraordinary pain possible. And there is no greater joy or pain life can deliver than that you get from your children. Sweet and sad, poured from the same vessel in equal measure.

What motivates you to inspire self-care in others?

I lost my eldest brother Randy in 1987 to suicide. Since then, I lost my brother David in 2011 to a cerebral haemorrhage, and my baby brother Michael in 2013, to a heart attack. In 1998 I was given a head and neck cancer diagnosis. My interest in self-care, kindness and wellbeing is not a casual interest, it has to do with me, it has to do with the people who I love, both the ones who I know, the ones I’ve lost, and the ones who I don’t know yet.

Kirk Strosahl

Maybe he is just being flippant, but I was uncomfortable with the strong, but implicit suggestion of Kirk’s title that people chose to suffer.

Kirk pain is inevitable

How would you describe ACT in a nutshell?

I think ACT tries to promote people who are open to their own experience and can separate themselves from the literal meaning of their experience – so that they’re not governed by thoughts or feelings or emotional reactions to things. They are in touch with their personal values so that they are geared toward living life to its fullest.

What are your most inspired actions?

I think the time I truly felt the most inspired simply in terms of output was writing my most recent book, Inside This Moment. I was in a zone for the entire time that I was writing it, it was coming from the heart. It was actually quite an amazing experience. The book zeroes in on how to use the present moment in therapy to inspire people to live their lives to the fullest, and not to run from their own demons, but rather cradle them. So that was enjoyable. Then I’ve also always been inspired to work with Patti [Dr Patricia Robinson]. We have a lot of fun and there’s intellectual growth when we write and get our arms around stuff together. So I’ve been very lucky that way.

Where are ACT and the other so-called ‘third-wave therapies’ heading now?
I think people are going to have to get their arms around what we mean by ‘mindfulness’ in a much more scientifically sound sense. Not the term, not the popular concept, but understanding exactly what goes into it. We’re still very overly general about it in our conversations, and because of that we’re losing leverage in therapy that we would otherwise gain by being more discriminating. That’s one area I think is going to see a lot of further growth. And I think we’re going to be looking at how we get our treatments briefer without losing effectiveness, because resource systems out there simply can’t afford the longer-term version of CBT or ACT for the masses – that’s another big area. How are we going to populate these concepts into public health models and work with lay people, primary care providers and teachers, for example? We’re still very therapist-centric in our profession, and because of that our population health effectiveness has been extremely limited.

I think the other area that is going to become bigger and more important is values-based behaviour change. It’s not like people haven’t been exploring it, but I think the amount of development will increase. There are measurement issues that are going to have to be dealt with, as well as creating more efficient ways of talking to people about motivating factors in their lives.

Kirk is modest in his aspirations for ACT, but I don’t see the relevance of evidence.

What is the greatest opportunity ACT has in today’s world?

I actually think that, if we don’t fall on our own sword, it’s going to be about bringing mindfulness concepts into the general public and Western civilisation. That’s not going to mean getting everybody in the West to put in hours of practice a day, sitting on a pillow ‘umm-ing’. Rather it’s this idea that these are actually pretty portable interventions based on neuroscience. They have a very rapid effect on brain neural pathway development and brain efficiency, and they don’t have to be these onerous, lifelong practices. There’s going to be a huge opportunity there if we can get the right message to people – that these are things you can teach yourself and your brain in small bits, and that it’s more about persistence than the amount of time you take. It’s about doing things intentionally and practising intention, as well as practising paying attention. These two things go hand in hand in mindfulness – this ability to pay attention in a particular way and then to act with intention inside of your own space. To me those things are so intricately linked to psychological health that if we could get those out into the public domain in ways which didn’t seem overwhelming to people, that would be a huge accomplishment for ACT, or for any of the mindfulness-based therapies that could get this figured out. So it isn’t just in the hands of a few peopl

Are ACT and CBT actually little more than saying to people ‘live with it, or change it’?
There are hidden properties of treatments, and then there are the observable properties of treatments. In ACT the observable properties are quite different, with an emphasis on values. It’s saying to people ‘don’t just tolerate your life, build your life from within’. ACT uses values as a foundation for addressing that.

Then there’s the ability to create space between you and what starts to show up in your life when you start doing things that matter. It’s an optimistic treatment that assumes people can do amazing things if they get lined up behind the right psychological processes. And it might well be that in CBT, even though ostensibly focusing on helping people change thoughts and behaviours, the act of talking about thoughts and behaviours is in a way itself a kind of a ‘defusion’ intervention. That’s what I mean by ‘hidden’ properties. You think that the mechanism is about the client becoming more logical and less irrational, but it may in fact be that by talking about thoughts you’re actually doing ‘defusion’ without even realising it.

I am also uncomfortable with the message that effective therapy can be delivered in 15 minutes. I am sure it sells well in behavioral health settings or in neoliberal societies where the goal is to underfund mental health services, but claim to offer them broadly. But strong claims with such potentially socially pernicious implications need very strong evidence. I have searched and can find none.

You’re a specialist in delivering brief, or ‘focused’ ACT interventions to patients – sometimes as brief as 15 minutes. Could you give a picture of what you might focus on in a session, say for diabetes or depression?
The goal of focused ACT is to get patients to make direct contact with the unworkable results of their current life strategies. Usually, these strategies involve avoiding dealing with important life issues – such as maintaining social health, managing diet or other health risk behaviours in the case of diabetes. The counter-weight in focused ACT is to get the patient to make direct contact with what matters to them in their life, and whether their avoidance behaviours are helping them move in that direction. This discrepancy creates a ‘healthy anxiety’, which we encourage patients to accept as a ‘signal’ that some type of change in personal strategy is needed. You don’t get people to change behaviours by giving them a label, or scaring them with adverse consequences if they don’t change, or lecturing them about the necessity of change. Change comes from within, not from without. Most patients know implicitly that they are avoiding things, but they don’t want to be condescended to, criticised or cajoled about it. So focused ACT is a very humanising approach in which we readily agree that making important changes in life might likely trigger painful emotional consequences or distressing memories of past failures, et cetera. Pain is inevitable, but suffering is optional.

The therapist and patient are on the same journey in this, and they’ve just happened to run into each other. There is no difference really between us.

Steve Hayes

Last, but certainly not least, Steve’s interview is titled with a rhetorical question that he does not get around to answering.

steve why can't we

How do your ideas connect with your values?

I’ve tried in my career to be less concerned about brand names and personal applause, and more concerned about building community and connecting to this larger value of ‘how can we best serve people using science?’. I get more excited in my personal work about supporting the work of others. I mean I’m productive, I do research, I write books. But to this day I’ve never not responded to an email. If a patient contacts me and says, ‘I’m in misery, what do I do?’: yeah, it might be two or three sentences, but I’m going to try and do something. So my value is being of use and building a community that can profoundly be of use – trying to bring science into alignment with what society needs and wants from us. And I don’t think we’ve always served that.

Surely, you jest, Steve.

You’re one of the founders of acceptance and commitment therapy, or ACT. What is the greatest opportunity ACT has in today’s world?

Reaching the culture. My son’s favourite cartoon show is Steven Universe – it’s very sweet. Garrett is a kind of a meditation or mindfulness person, a wise adult. She sings a song called ‘Here comes a thought’, and the show notes say she got it from ACT. The first stanza is ‘flexibility, love and trust’, and it walks through what to do with difficult thoughts. It talks about seeing and feeling, and watching the thought. And they actually lie on the ground like they’re watching clouds. I started crying seeing it because I’m thinking, ‘how many children have shame that they’re carrying?’ It’s got to be a large percentage of children, right? And they can’t talk to their parents, and they can’t talk to their peers, because what is it they’re going to say? ‘Just don’t think about it.’ And it’s like pouring gasoline on a fire, it’s just a horrible thing to do to shame… it’s saying there’s something wrong with you for feeling and thinking that, and you just need to think differently and it’ll go away. We can do something a lot wiser than that.

When I teach brand new students, say, research methods, the very first thing I get across is this: the modal number of citations for a publication in psychology is zero, which means nobody has been influenced by it enough to actually put in writing that it mattered. So I only want something that you really have heart for, something that deep down you think might really matter. And then I tell them, ‘and by the way, your research idea almost certainly is a bad idea’, because that’s how it [the modal number of citations] gets to zero. From there we can figure out a way that maintains the heart, and that won’t have zero citations. There’s like this dialectic of ‘how do we come into our field as social human beings with this core of caring, and nurture and sustain that, but also learn how to channel our interest and questions in a way that will have a long term impact on the world – on our clients and on our colleagues?’.

What’s something people don’t know about you, that if they knew, they would be surprised?
I think people are sometimes surprised that if I wasn’t going to be a psychologist, my other big thought about what I would be is a carpenter. I’m a veteran re-modeller, I’m tearing down my house all of the time. People sometimes think I’m a geek, but I can put in plumbing and I’ve built an entire house from nothing, from the foundation to the roof – done everything myself. So I’m a builder and I try to bring that sensitivity into what I do research-wise and clinically also. I want to build something that lasts and that matters, and I think we’ve been able to do it in some of the work we’ve been doing in CBT and ACT, and Mindfulness and ACBS etc.

I can’t add to this. I will just let Steve speak for himself.

What to look for in the Special Issue of Journal of Health Psychology concerning the PACE trial

special issue spread

 

A Special issue of Journal of Health Psychology concerns PACE, a trial of therapies for patients with myalgic encephalomyelitis (ME)/chronic fatigue syndrome (CFS) that has attracted a great deal of controversy

Key points summarized from the David F. Marks’ introductory editorial

 Background

The idea for Special Issue started with Journal of Health Psychology receiving a manuscript from Keith Geraghty providing a critical review of the PACE Trial.

Following peer review and acceptance of a revision, the PACE investigators were offered an opportunity to respond to Keith Geraghty with an Open Peer Commentary paper.

The PACE investigators made a number of efforts to get Geraghty’s article outright retracted, either directly or through pressures on the editorial board.

When it was clear that the journal would not cave to such pressures, the  PACE investigators then demanded a partial retraction of the Geraghty  article and that the journal issue a correction reporting that the author had an undisclosed conflict of interest: he had failed to acknowledge that he was a patient suffering from myalgic encephalomyelitis.

The PACE investigators also demanded that their commentary manuscript will not be subject to peer review.

The PACE investigators received reviews of their manuscript. Editor Marks notes [Having been involved in editing this issue, I agree]:

After receiving critical reviews, the pro-PACE authors chose to make only cosmetic changesor not to revise their manuscripts in any way whatsoever. They appeared unwilling to enter into the spirit of scientific debate. They acted with a sense of entitlement not to have to respond to criticism.

The review and response from the PACE investigators was sent to more than 40 experts on both sides of the debate for commentaries.

After the online publication of several critical Commentaries, the PACE investigators were offered a further opportunity to respond to their critics in the round but they chose not to do so. There was little response, but a notable declining of further comment by the PACE investigastors:

As always, we would refer interested readers to our original publications and trial websitewhere most, if not all, the issues brought up by commentators are addressed

Conflict of interest.

[My comment: The PACE investigators and persons associated with them were quick to complain about the conflict of interest in the developing issie. As noted, this including a complaint about Geraghtybeing a patient and not disclosing it, mentioned above. But the investigators also threatened a formal complaint to the Committee on Publication Ethics concerning my having reviewed their manuscript. But there is an ironic backstory about the matter of conflict of interest. Some of which are discussed in contributions to the Special Issue:]

The PACE authors themselves appear to hold strong allegiances to cognitive behavioural therapy (CBT) and graded exercise therapy (GET) – treatments they developed for ME/CFS. Stark COI have been exposed by the commentaries including the PACE authors themselves who hold a double role as advisers to the UK Government.

 Department of Work and Pensions (DWP), a sponsor of PACE, while at the same time working as advisers to large insurance companies who have gone on record about thepotential financial losses from ME/CFS being deemed a long-term physical illness.In a further twist to the debate, undeclared COI of Petrie and Weinman (2017) werealleged by two of the commentators (Agardy, 2017; Lubet, 2017). Professors Weinman andPetrie adamantly deny that their work as advisers to Atlantis Healthcare represents a COI.

[My comment: In another instance, three pro-PACE authors attempted to subvert the journal’s policy on COI by recommending reviewers who were strongly conflicted, forcing rejection of their paper. I discussed this in another blog post.]

[My comment: Nonetheless, having gotten any commentary from the PACE investigators at all was an achievement. Even the limited responsiveness of the PACE investigators to Geraghty’s commentary, along with a supported commentary by their close colleagues Keith J Petrie and John Weinman, the special issue  represents one of most significant exchanges between defenders of the PACE trial and critics in a long while.]

ME/CFS research has been poorly served by the PACE Trial and a fresh new approach is needed by the PACE investigators to engage critics and skeptics.

Overall, the special issue provides, readers to make up their own minds about the scientific merits and demerits of the PACE Trial.  

Note that the entire Special Issue is available with free access.

journal of health psychologyThe individual papers

‘PACE-Gate’: When clinical trial evidence meets open data access. Keith J Geraghty

Published reports of the PACE trial overstated the effectiveness of cognitive behavioural therapy and graded exercise therapy, and did so  by lowering the thresholds they used to determine improvement. Reanalyzes conducted by professionals and patients revealed that the treatments tested had much lower efficacy after an information tribunal ordered the release of data from the PACE trial to a patient who had requested access using a freedom of information request.

Response to the editorial by Dr Geraghty. Peter D White, Trudie Chalder, Michael Sharpe, Brian J Angus, Hannah L Baber, Jessica Bavinton, Mary Burgess, Lucy V Clark, Diane L Cox, Julia C DeCesare, Kimberley A Goldsmith, Anthony L Johnson, Paul McCrone, Gabrielle Murphy, Maurice Murphy, Hazel O’Dowd, Laura Potts, Rebacca Walwyn and David Wilks

The trial found that adding cognitive behavior therapy or graded exercise therapy to specialist medical care was as safe as, and more effective than, adding adaptive pacing therapy or specialist medical care alone. Dr Geraghty has challenged these findings. We suggest that Dr Geraghty’s views are based on misunderstandings and misrepresentations of the PACE trial.

Once again, the PACE authors respond to concerns with empty answers. David Tuller

This commentary examines how the current response once again demonstrates the ways in which the investigators avoid acknowledging the obvious problems with PACE and offer non-answers instead—arguments that fall apart quickly under scrutiny.

Investigator bias and the PACE trial. Steven Lubet

Standards for determining researcher bias are considered and it is concluded that the PACE nvestigators’ impartiality might reasonably be questioned.

The problem of bias in behavioural intervention studies: Lessons from the PACE trial Carolyn Wilshire

In the PACE trial, cognitive behavioural therapy and graded exercise therapy had modest, time-limited effects trial on self-report measures, but little effect on more objective measures such as fitness and employment status.  In non-blinded trials, the issue of reporting biases deserves greater attention in future.

PACE trial authors continue to ignore their own null effect. Mark Vink

Protocols and outcomes for the PACE trial were changed after the start of the trial,  leading to exaggerated claims for the efficacy for both cognitive behavior therapy and graded exercise therapy. Findings of small, self-reported improvements in subjective measures cannot be used to say the interventions are effective, particularly in light of the absence of objective improvements with objective outcome measures.

The PACE trial missteps on pacing and patient selection. Leonard A Jason

The PACE trial investigators were not successful in designing and implementing a valid pacing intervention and patient selection ambiguity further compromised the study’s outcomes

Do graded activity therapies cause harm in chronic fatigue syndrome? Tom Kindlon

The trial’s poor results on objective measures of fitness suggest a lack of adherence to the activity component of these therapies. Therefore, the safety findings may not apply in other clinical contexts. Outside of clinical trials, many patients report deterioration with cognitive behavioural therapy and particularly graded exercise therapy. Also, exercise physiology studies reveal abnormalities in chronic fatigue syndrome patients’ responses to exertion. Given these considerations, one cannot conclude that these interventions are safe and risk-free.

PACE team response shows a disregard for the principles of science. Jonathan Edwards

The response by White et al. fails to address the key design flaw, of an unblinded study with subjective outcome measures, apparently demonstrating a lack of understanding of basic trial design requirements. The failure of the academic community to recognise the weakness of trials of this type suggests that a major overhaul of quality control is needed.

Bias, misleading information and lack of respect for alternative views have distorted perceptions of myalgic encephalomyelitis/chronic fatigue syndrome and its treatment. Ellen Goudsmit and Sandra Howes

These interventions delivered in the PACE trial are based on a model which assumes that symptoms are perpetuated by factors such as misguided beliefs and a lack of activity. Our analysis indicates that the researchers have shown significant bias in their accounts of the literature and may also have overstated the effectiveness of the above treatments. We submit that their approach to criticisms undermines the scientific process and is inconsistent with best practice.

PACE investigators’ response is misleading regarding patient survey results. Karen D Kirke

A review of survey data published between 2001 and 2015 reveals that for most patients, graded exercise therapy leads to worsening of symptoms, cognitive behavioural therapy leads to no change in symptoms, and pacing leads to improvement. The experience of people with ME/CFS as reflected in surveys is a rich source of information, made more compelling by the consistency of results. Consequently, patient survey evidence can be used to inform practice, research and guidelines. Misrepresentation of patient experience must be vigorously challenged, to ensure that patients and health professionals make decisions about therapies based on accurate information.

 

 

 

 

Global expert on distant and faith healing chaired PACE Data Monitoring and Ethics Committee

mild distance-healing1Paul Dieppe, the initial Chair of Data Monitoring and Ethics Committee for the PACE trial is also renowned as a “leading global voice in the understanding and advancement of energy/ spiritual healing.”

Most of my medical colleagues think I’ve gone completely barmy –  Paul Dieppe

Renowned as a leading global voice in the understanding and advancement of energy/ spiritual healing

Paul Dieppe, the Chair of Data Monitoring and Ethics Committee of the PACE trial of cognitive behavior therapy and graded exercise therapy for chronic fatigue syndrome has had a distinguished career, up to a point. He has emeritus status at the University of Exeter Medical School and is described on the website:

Paul qualified as a doctor in London in 1970. He specialised in rheumatology and became ARC professor of rheumatology in Bristol in 1987, and Dean of the Bristol Faculty of Medicine between 1994 and 1997. He then switched to health services research and was the Director of the MRC Health Services Research Collaboration between 1997 and 2007. After that he spent a short time at the University of Oxford before moving to the Peninsula Medical School to work in clinical education research.

With the formation of the University of Exeter Medical School, he took on the role of Professor of Health and Wellbeing.

Yet a more recent (2016) website gives a different picture of him:

exeter professor discovers

And gives a chance for  Dieppe to describe himself:

Paul, now renowned as a leading global voice in the understanding and advancement of energy/ spiritual healing, said: “My goal is to work towards the greater understanding that we can bridge the gap between biomedical thinking and the world of unexplained phenomena around areas such as healing, which if we placed greater value upon, could play a crucial role in the wellbeing of society.”

A further advertisement for the conference at which Dieppe spoke stated that there would be live demonstrations of healing on stage and describes his talk as:

The incredible power of remote healing across distance:  Paul Dieppe, emeritus professor of health and wellbeing, renowned for being at the global forefront of research into Healing.

faith healer in NHSA press release for “the largest meta-analysis of ever undertaken into the effects of non-contact healing”  states:

In fact, there are many healing practices where healer(s) and the one needing healing never come in contact with one another.  Indeed, they may never even meet, let alone know each other.

Paul Dieppe is identified as a Trustee of the group that produced a meta-analysis and he comments on the findings:

“This is a rigorous, high quality scientific report, and it clearly shows that healing intention can have beneficial effects on living systems, both human and non-human. Now we need to explore questions such as who does it work for and in what circumstances, as well as the how and why questions.”

A 2015 interview shows him rejecting scientific methodology as old fashioned materialism. Instead as he favors  anecdotes  and embraces energy/spiritual healing.

Have you come across many instances of actual healing?

Masses of cases. It’s anecdotal stuff, but it’s extremely impressive and extremely convincing. I’ve experienced it myself. A lot of this stuff happens completely under the radar, people just getting on with it in their back room, not part of any particular body or group, just doing their own thing. I went to see one such person, a little old lady in her front-room, and she said the best way to understand healing is to experience it yourself, and asked if I had anything I wanted working on. I told her I had a bad knee, so she said, OK let’s give it a go. And for what it’s worth, it’s been better since then. That was a couple of years ago.

Do you have a sense of what happens in the brain, does it involves specific neural or nervous networks? What systems does it involve?

Life gets difficult when you talk like that. You’ve slipped into the assumption that our materialist knowledge is the appropriate framework for that. We assume our current materialist science can explain everything, so the default position is ‘how can I explain this physiologically within my own materialist framework?’ I’m not convinced that’s the right way to do it.

No, I don’t have a materialist worldview, but if there is a spiritual dimension (which I think there is) I’m curious as to how it interacts with the body. William James, for example, explored how spiritual experiences interacted with ‘the subliminal self’. I wonder if healing is connected to things like trance states, altered states of consciousness, and so on.

OK, well how might it happen in those terms? Certainly there’s a lot of evidence for our ability to alter things through the Autonomic Nervous System via hypnosis. That can give us clues to a lot of this stuff. Hypnosis certainly affects the ANS, it certainly affects the immune system…probably everything. The mind / body split is of course silly. Everything is connected, everything works together. Although it’s easiest to talk in terms of what we can observe physiologically…I think we can control pretty much all of it probably.

OK, so at the moment we’re at or near the peak of a biomedical conception of health and illness. You, by contrast, espouse what you describe as ‘a bio-psycho-socio-spiritual theory of the transcendence of suffering’. How does that go down in academic medical circles?

Most of my medical colleagues think I’ve gone completely barmy, and this is all absolute nonsense. The dominant model in which we work and teach rejects anything to do with spirituality. If you mention words like spirituality or love, you’re rejected. It’s not acceptable behaviour, you’re regarded as someone who should be quietly taken off to the funny farm. So it’s quite lonely. It doesn’t bother me, I’m at the end of my career, I don’t need a new job or a reference, I don’t need to toe the line. It’s irrelevant to me. But I can feel the group discomfort sometimes around this area. I think the medical profession is a bit out of kilter with the rest of society, which is probably more accepting of the link between spirituality and health.

The interview links to a video featuring Dieppe on a trip to Lourdes, a destination for millions seeking miraculous cures.

The Role of the Data Monitoring and Ethics Committee in a Clinical Trial

As chair of the PACE DMECT, Paul Dieppe was in a unique and powerful position, able to monitor incoming data that was unblinded with respect to the group to which participants were assigned.

A recent Medical Research Council document states

6.2 DATA MONITORING COMMITTEE

The role of the Data Monitoring Committee (DMC) is to monitor the data emerging from the trial, in particular as they relate to the safety of participants, and to advise the Trial Steering Committee on whether there are any reasons for the trial not to continue. It is the only body involved in the trial that has access to the unblinded (unmasked) comparative data during the trial.

At the time of the fateful shifting of scoring of subjective outcomes and downgrading of the status of objective outcomes, Dieppe more than anyone else was in a position to know how switching would effect results of the analyses of the data already collected. He supposedly reviewed analyses in closed meetings, excluding Principal Investigator Peter White, but he was also meeting White formally and informally.

distasnce healingDieppe knew exactly what was going on in the most recent data, in terms of emerging  patterns of group differences. He would also have attended the meetings in which changes to the protocol were discussed and approved. Did he remain quiet? Refuse to meet with White in private? Maybe Dieppe relied on the unspoken interconnected consciousness of the two men.

We have a mystery akin to that about what when on before the US elections between Trump and his advisers, on one hand,  and the Russians.

The significance of a science-skeptic heading the PACE DMET

 Even before Dieppe’s colleagues would have described him, as he puts it, going completely barmy, he was a believer in the extreme, almost mystical power of the placebo. Perhaps that made him an ideal choice to head the DMET for a trial that created such an imbalance in nonspecific effects between the active treatments and the comparison group. And then boosted the imbalance as part of treatment and even in a newsletter (!).

But seriously, what we make of someone having such disdain for scientific methodology having the responsibility for ensuring its rigor in a major clinical trial?

 

 

 

 

What?! Homeopathy and acupuncture for symptoms of autism

giphy

 

Let’s get the American Association for the Advancement of Science (AAAS) to stop publicizing junk studies of acupuncture for symptoms of autism intended to exploit parents NOW.

 

 

Changes are ordered by Advertising Standards Authority for Ireland (ASAI) in the Facebook Page advertising acupuncture for autism.

  • Yet, an enterprising journal editor got the AAAS’s Eurkalert to distribute worldwide a press release for a junk study of acupuncture for autism that his journal published.

  • Some lessons and practice in skills in detecting junk studies.

  • Let’s recognize The Journal of Alternative and Complementary Medicine (JACM) as a Quack! purveyor of junk promotions of discredited and unproved treatments disguised as science.

  • Let’s watch for the next move of the Editor of JACM’s next move in his avowed effort to gain legitimacy and reimbursement for acupuncture.

  • Let’s tell AAAS to stop making excuses.

  • Tell AAAS to get EurekAlert to screen press releases before distributing them. As it stands, EurekAlert is damn confusing to lay persons who consider the AAAS bran attached to anything as indicated a trusted source. [At the end of this blog, I make it easy for you to lodge a protest.]

Ireland’s Advertising Standards Authority investigated a complaint. As a result, the ASA has informed a homeopath not run a Facebook advertisement in its current form.

The homeopath was told by ASAI not to run a Facebook advertisement in its current form again because the page suggests she could treat the symptoms of autism. The ad by Almond Homeopathy stated the homeopath had:

“the great pleasure of meeting and working with many children diagnosed with ASD and the greater pleasure and joy of helping them overcome many of their symptoms that were holding them and their families back from getting the most out of life.”

The Facebook Page also referenced a blog which contained a reference to a child whose “learning, relationships and speech” had improved

The advertiser defended her FB page by declaring  that  she was a qualified homeopath, registered with the Irish Society of Homeopaths.

She said she treated the person, not their condition or disease, and did not claim to be able to cure anything or anyone.  However,

The ASAI’s complaints committee said that, while noting the advertisers’ comments that she was a qualified and licensed homeopath and registered with the Irish Society of Homeopaths, it was aware from a previous case that the Department of Health had confirmed that there was no statutory regulation of complementary therapists.

The committee also said the wording of the post and the heading of the blog post would give readers the impression she could treat the symptoms of autism.

About the same time, EurekAlert’s posted on Twitter a link

New study from KKI shows feasibility of acupuncture in young children with ASD

To the naïve reader, the item appeared to be a story, not  the pasted in press release that it was, but that was not the case. Where one would have expected the name of a journalist, the publisher’s name was listed.

MARY ANN LIEBERT, INC./GENETIC ENGINEERING NEWS

New Rochelle, NY, June 19, 2017-A pilot feasibility study to determine if young children with autism spectrum disorder (ASD) and their parents would tolerate and adhere to an office- and home-based acupuncture/acupressure intervention showed completion of all 16 biweekly sessions and measurements of their effects before, during, and after the protocol. The study design and results, which suggest further controlled studies of this intervention approach in ASD, are published in the Journal of Alternative and Complementary Medicine (JACM), a peer-reviewed publication from Mary Ann Liebert, Inc., publishers. The article is available free on the JACM website until July 19, 2017.

Coauthors Lana Warren, EdD, OT/L and Patricia Rao, PhD, Kennedy Krieger Institute, Baltimore, MD, and David Paton, DAc, LAc, Starting Point Acupuncture and Health Services, Catonsville, MD, identified the most positive outcome of the study as the high compliance rate, with all parents of the children ages 3-10 completing the intervention. The researchers measured the effects of the intervention on factors such as the children’s behavior, ability to pay attention, sleep, and aspects of parenting stress. In the article entitled “A Pilot Observational Study of an Acupressure/Acupuncture Intervention in Children with Autism Spectrum Disorder,” most parents reported that the intervention had a positive impact on their relationship with their child.

The editor of JACM chimed in:

“While a small study, the tolerance and adherence with acupressure this pilot are both hopeful signs for families of those in their care with autism spectrum disorder,” states JACM Editor-in-Chief John Weeks, johnweeks-integrator.com, Seattle, WA.

Before getting to the horrible little study itself, note how the AAAS  denies responsibility for what it posts:

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.

Note also that AAAS has said it employs 20 staff to monitor its presence in social media and tweak publicity.

The “pilot study” was:

Warren Lana R., Rao Patricia A., and Paton David C. A Pilot Observational Study of an Acupressure/Acupuncture Intervention in Children with Autism Spectrum Disorder. The Journal of Alternative and Complementary Medicine. May 2017, ahead of print. https://doi.org/10.1089/acm.2016.0257\

The Abstract

Objectives: To determine whether children with autism spectrum disorder (ASD) would tolerate an acupressure/acupuncture intervention and whether parents would adhere to a twice-weekly, 8-week intervention protocol. Second, to further understand best measures to use to capture impact of intervention on behavioral and regulatory functions.

Design: This is an observational pilot study with pre-, mid-, and postintervention measures.

Settings/location: The intervention was carried out in a private practice office in a large metropolitan area.

Subjects: A total of 10 children of ages 3–10 years with ASD and one of their parents participated.

Interventions: A total of 16 biweekly treatment sessions of acupressure and/or acupuncture were carried out by a licensed acupuncturist, and a daily home-based acupressure intervention was carried out by a parent.

Outcome measures: Attendance, tolerance of intervention, parent compliance with home program, and parent compliance in completing daily diary and five standardized measures of behavioral and regulatory functions pre-, mid-, and postintervention were recorded.

Results: The 10 children in this observational study, collectively, tolerated the intervention and parents adhered to the 16 sessions, biweekly protocol, and home protocol, as well as completing daily diary and five standardized measures at three different time intervals. The five measurements appeared to be sensitive to behavioral and regulatory functions that may improve with this type of intervention.

Conclusions: The results of this observational pilot study suggest that acupressure/acupuncture is a feasible intervention for children with ASD that merits rigorous evaluation through a randomized controlled trial.

Probing the article

Keep in mind:

There is no evidence that acupuncture is anything but an elaborate placebo ritual. The ritual involves efforts to communicate hope and acupuncturists are trained to express a sense of progress when patients wince or express discomfort.

There is no specific scientific mechanism by which acupuncture could be expected to affect physical and mental health outcomes.

This is not a randomized trial. Any changes observed  could be chance variation or return to the usual behavior of the patient from the whatever increase in problems motivated seeking health.

Outcomes are not objective, but depend on the subjective ratings from parents who have invested time in repeated traveling to the clinic where treatment is provided. The parents are not blinded, they know their child is receiving a treatment that the clinic communicates is effective. There is no comparison-control group. It is a very different situation than getting the same ratings from teachers who know the children, but do not know if they have received treatment.

A sample of 10 is inadequate (underpowered) for any claims of efficacy or effectiveness. In a randomized trial, a sample of 30 patients per group is needed to detect half the time when the effectiveness of a  treatment that actually has moderate-sized effects. Half the time the effects of the treatment would be missedWhen statistically significant results are found, they are likely to be flukes. Any results that reached significance would have to be a large difference. The literature on psychotherapeutic interventions for mental health problems is littered with unpowered studies claiming significant effects that do not replicated in larger studies.

Why is that? There is a strong confirmation bias in what gets published. Expectable null findings do not get published. The positive effects from small studies that do get published are likely to chance, selective reporting, or the researchers exercising some selection and control of what is reported that they should have.

The ability to detect moderate sized effects drops off quickly as sample size decreases.

A sample of 10 patients would need huge effects. Most statistics do not perform well with such a small sample.  Sampling error or just plain noise affect results: Loss of one patient or adding of another could dramatically affect results. But statistical analyses depend on assumptions being met about the distribution of data. These assumptions are not likely to be met with 10 patients and could not even be tested.

The bottom line: Competent, well-meaning psychotherapy researchers do not even claim any evidence a treatment is effective from a sample  of 10. Authors who make such claims are fools or are trying to fool their readers , or both.

An added bonus guide with helpful tips from a source I trust

Small studies: Be vigilant when writing about them and skeptical when reading about them – HealthNewsReview.org

Nonetheless, the authors make some dubious claims

Unexpectedly, the most robust finding of the current pilot observational study was significant improvement in children’s expressive and receptive social communication after the intervention. These findings are consistent with previous studies reporting significantly improved social communication after acupuncture, including language comprehension and social interaction.14,18 However, they were unexpected because the acupoints selected for intervention in the current study were chosen to target areas of the body corresponding to behavioral and regulatory problems, not areas known to be involved with social communication.

Beware of unexpected robust findings from such little studies. The authors admit that they were not  stimulating the bogus acupuncture points that are supposed to produce such change. They cite crap studies that nonetheless suggest such results should be found.

Although not statistically significant, the current findings indicated some improvement in sleep delay after the intervention, which may be worth further investigation.

Sure, the authors would be better off consulting tea leaves or the children’s horoscopes. They are sorting through lots of measures searching for other significant ones to crow about. When they come up  empty handed, they declare success and call for more research.

Of note, sensory processing concerns associated with ASD, particularly tactile, which had been anticipated to be problematic in AIs, did not manifest in the child participants in this study, with the exception of their general dislike of maintained deep pressure in one location. It should be noted that the initial acupressure strokes administered during the intervention were those that promote calming and quieting per Chinese Medicine theory. response to treatment.

Let’s go back to what the editor of this journal said in the press release:

“While a small study, the tolerance and adherence with acupressure this pilot are both hopeful signs for families of those in their care with autism spectrum disorder,” states JACM Editor-in-Chief John Weeks, johnweeks-integrator.com, Seattle, WA

Quack, Quack,

 

Any journal that would publish such irresponsible stuff deserves a warning to ward off naïve and vulnerable readers. Nija Ezard Ernst says

I only see one solution: completely disregard certain journals that have been identified to regularly publish nonsense. Sadly, the wider medical community is far from having arrived at this point. As far as I can see, the problem has not even been identified yet as a serious issue that needs addressing. For the foreseeable future, we will probably have to live with this type of pollution of our medical literature.

The JACM Editor-in-Chief John Weeks is well known to the skeptical community who watch for quacks. Like the EverReady Bunny, he jumps at opportunities to promote the reimbursement of dubious, unproven treatments. For instance:

The Academic Consortium for Complementary and Alternative Health Care (ACCAHC) is a group of “complementary and alternative medicine” educational organizations representing chiropractors, naturopaths, acupuncturists, homeopaths, ayurvedic practitioners, direct entry midwives and massage therapists. The executive director is John Weeks, a relentless promoter of “integrative medicine,” both on the Integrator Blog website and in the Huffington Post. The ACCAHC is dedicated to ensuring that its members and the practitioners they represent are included in all aspects of health care, such insurance reimbursement, financial resources for education and delivery models. ACCAHC’s stock-in-trade is its practitioners’ supposed expertise is being patient-centered, holistic, taking into account the whole person and such, as well as an alleged emphasis on healthy lifestyles, nutrition, well-being, and the like.

AAAS puts this stuff out there because it makes money. You can tell them to stop. Contact them at their website, go to their Facebook page. Share and Tweet the link to this blog post, tweet to EurekAlert.

EurekAlert is promoting quack treatments to vulnerable parents of children with autism and it must stop. Readers may not notice that distributing through the EurekAlert doies not constitute endorsement by the widely respected Association for the Advancement of Science, despite the disclaimer at the bottom of the page.

Contact EurekAlert!

E-mail: webmaster@eurekalert.org

Phone: 1-202-326-6716

Fax: 1-202-898-0391

Address:

EurekAlert!/AAAS

1200 New York Avenue, NW

Washington, DC 20005

USA

Twitter: @EurekAlert

Facebook: facebook.com/EurekAlert

 

A skeptical look at a study of acupuncture delivered in emergency rooms [updated]

Why an expensive, flawed, misrepresented trial of acupuncture in emergency rooms provided no evidence that acupuncture is effective in controlling pain.

acupuncture.needlesUpdate 6/24/2017

I  thank the pseudonymous commentator who called attention to one of the authors of this study previously providing a press release making claims for the study’s results before the data were analysed   The author stated:

“While data from the study is still being analysed and finalised for publication in a medical journal, one of the researchers, Dr Michael Ben-Meir, said it showed acupuncture offered the same level of pain relief as analgesic drugs when patients rated their pain one hour after treatment.”

We can now evaluate this misrepresentation.

A number people in social media urged me to take a quick look at a recent large, expensive trial of providing acupuncture in the emergency room. The study has begun getting some attention in the media. We will see the study is flawed from its inception, aimed at fulfilling the principal investigator’s agenda being of misrepresented as supportive of getting acupuncture into ERs. Coverage in the media is inaccurate, but consistent with statements made by the primary investigator.

The paper reporting the clinical trial is unfortunately in a pay-walled journal, but it is at least temporarily it is available here.  An abstract can be found here

Marc M Cohen, De Villiers Smit, Nick Andrianopoulos, Michael Ben-Meir, David McD Taylor, Shefton J Parker, Chalie C Xue and Peter A Cameron Acupuncture for analgesia in the emergency department: a multicentre, randomised, equivalence and non-inferiority trial. Medical Journal of Australia. 2017; 206 (11): 494-499. || doi: 10.5694/mja16.00771

The trial protocol is freely available and can be found here.

  • My summary assessment is that it is indeed an ambitious expensive clinical trial producing no evidence that acupuncture is effective.
  • Its rationale is based on misrepresentations of the literature.
  • The study protocol shows features are built in its design to be biased in favor of finding acupuncture is not inferior to routine care in the ER.
  • The article downplays important findings that undercut validity of trial and investigator’s intended conclusions.
  • The study was conducted by an investigator strongly committed to the promotion of acupuncture in routine medical care and securing reimbursement for it.
  • The conclusion is slanted toward cultivating the illusion that acupuncture is an evidence-based treatment. The intent is to obtain funding for an even more ambitious trial, ultimately getting acupuncture secure reimbursement and preferred treatment status.
  • The study capitalizes on the current opioid addiction epidemic and public health concerns about overreliance on pain medications.

Rationale for the randomized trial

The article and protocol seriously misrepresent the conclusions of the existing literature. Claims that are made are inconsistent with was is found in the specific citations that are provided.

The article states

Acupuncture can provide analgesia for chronic musculoskeletal pain and headache,6-9 and its use is covered by therapeutic guidelines for acute pain.10,11

And

Our study aimed to determine whether acupuncture is effective, safe, acceptable and feasible for patients presenting to the ED with low back pain, migraine or acute ankle injuries.

The protocol states

It has been shown that acupuncture analgesia in the treatment of chronic pain is comparable to morphine and that its better safety profile and lack of dependence makes it the preferred method of choice for these conditions [9].

Reference 9 in both sources is to

Ernst E, Lee MS, Choi TY. Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. PAIN®. 2011 Apr 30;152(4):755-64.

This article is highly critical of the use of acupuncture and certainly does not say its effectiveness is comparable to morphine and should be the preferred method of treating chronic pain. Instead, it says:

In conclusion, numerous systematic reviews have generated little truly convincing evidence that acupuncture is effective in reducing pain. Serious adverse effects continue to be reported.

The protocol proclaims the pressing need for a study of the scope of the present one:

As of June 2011, virtually all of the 31 reviews and 32 review protocols listed in the Cochrane library suggest that the evidence for acupuncture is inconclusive. Consequently, there has been a call for the funding of well-planned, large-scale studies to assess the effectiveness and cost-effectiveness of acupuncture under real-life conditions [14]. It has been suggested that the best methodological approach to clinical acupuncture research is through pragmatic trials where acupuncture is compared to standard care rather than placebo-controlled trials. Such trials may be more able to provide data that is relevant to patients, practitioners and policymakers, and to inform decision-making about treatment options [15].

The issues about acupuncture are (a) the lack of a scientific rationale and (b) difficulty demonstrating it is superior to a sham acupuncture treatment. The difficulties in (b) is that it is difficult to blind a “real” versus sham treatment and the tendency of acupuncturists to communicate to patients whether they are receiving only a sham treatment.

There is a lack of evidence that acupuncture is anything but a highly ritualized placebo treatment. The investigator takes this as a cue for abandoning placebo-controlled studies and just comparing it with routine care- which is often lacking in the support and attention provided by a nonspecific placebo condition.

The investigator is intent on demonstrating that acupuncture is no worse- noninferior -to routine care.

Design features that  bias in favor of finding acupuncture is not inferior to routine care in the ER.

The emergency room (ER) is a bad place to get medical care. Don’t get your medical care unless required by circumstances.

The ER is also a bad place to evaluate acupuncture. There are reasons not to provide acupuncture in the ER even if one believes in the efficacy of acupuncture. ER care is urgent care intended to address medical issues needing immediate attention. For many medical conditions, emergency room care is only the first step in admitting a patient to inpatient services or to refer a patient for specialty outpatient services.

The conditions that are lumped together are too diverse, especially if the interest is the evaluation of the treatment of pain.

An ankle sprain is an acute injury. The article notes

All ankle injury patients received rest, ice, compression and elevation regardless of their treatment allocation.

Patients presenting to the ER with migraines are typically experiencing the exacerbation of a pre-existing chronic condition. Other symptoms besides pain may prompt a visit. The article notes that “migraine patients received intravenous fluids at the discretion of the treating physician.” The pharmacotherapy protocol included treatment for nausea, but no pain medication specific to migraines.

Patients presenting with acute low back pain may be injured and facing the onset of a chronic condition or facing an exacerbation of a chronic condition. Addressing the immediate circumstances of the ER visit is a goal, not analgesia. Long term goals are to preserve functioning and quality of life, not analgesia.

These three groups are different in their conditions and procedures, goals,  and expected outcomes for an ER visit. Even a trial as large as this one would be too small -underpowered- to explore crucial differences between groups. Or to see if overall results do not generalize to any patient group.

For all three patient groups, some will have self-limiting pain, which will contribute to a lack of differences among treatments.

There is no evidence for the efficacy of acupuncture versus a nonspecific treatment for any of these treatments. I recently discussed claims for acupuncture for migraines. I showed a lack of evidence for its efficacy. It is important to note, however, that in that study, acupuncture groups received treatment 5 days per week for 4 weeks for a total of 20 sessions. Even believers in acupuncture would not expect one session delivered in the ER would prove sufficient. But benign and even inert effects of one session might encourage patients to get more treatments later.

The primary outcomes of the study were assessed only at 24 and 48 hours. In those short periods, differences between effective and noneffective treatments may not have emerged. Differences in nonspecific treatments –attention, support, and communication of positive expectations may be maximized, but short-lived. And the subjective distress of some patients may have gotten dropped from the heights of distress which brought them to the ER.

Downplaying findings that undercut validity of trial and investigator’s intended conclusions

 Regardless of group assignment, all patients were entitled to “rescue medication” after 1 hour or if their attending physician deemed it appropriate. Patients assigned to acupuncture alone were three times more likely to receive rescue medication. This is a serious confound undermining the integrity of the trial and the meaningfulness of any results.

The acupuncture only group received significantly more rescue medication therapy than the groups that received pharmacotherapy (at T1: p = 0.016; after T1: P = 0.008).

Only a minority of patients in any condition achieved adequate pain relief.

Overall, 16% of patients had clinically relevant pain relief and 36.9% statistically relevant pain relief at T1, with no statistically significant differences between the three groups (Box 6).

Conducted by an investigator strongly committed to the promotion of acupuncture in routine medical care and securing reimbursement for it

 In Australia, the principal investigator is a sought after spokesperson advocating acupuncture and other unproven medical treatments.

Statements made in interviews with the media   show the motivation of the PI in conducting the trial and his spinning of his results:

“Emergency nurses and doctors need a variety of pain-relieving options when treating patients, given the concerns around opioids such as morphine, which carry the risk of addiction when used long-term.

And

“Our study has shown acupuncture is a viable alternative, and would be especially beneficial for patients who are unable to take standard pain-relieving drugs because of other medical conditions.

And

“But it’s clear we need more research overall to develop better medical approaches to pain management, as the study also showed patients initially remained in some pain, no matter what treatment they received.”

A sampling of the inaccurate coverage that this study received:

press coverage

Postscript

I enjoy probing suspicious studies like this one. I hope that readers benefit from what I find, not only in terms of my specific conclusions, but in acquiring the knowledge and skills I deploy. I draw on over 40 years experience as a researcher and a teacher, with over 400 peer-reviewed publications, as well as lots of editorial and peer review experience. In the case of this particular study of acupuncture, likely problems could be anticipated, as well as where to look for them.

Although I’ve becomes relatively efficient at this sort of activity, it remains time and labor intensive, particularly when I trace the study back into the claims being made about it in the media, as well as to any study protocol. And then look for discrepancies with what else is out there. And them format all of this for a blog post.

There is no direct benefit, professionally or financially to me for this activity. It’s an enjoyable, almost addictive distraction. I really should be working on the web-based science writing courses and e-books that will ultimately provide support for my activities, now that I am emeritus. I wonder, though, if there might be a way to monetize this. I would be uncomfortable simply doing it fee-for-service, because of the potential conflict of interest that it would create. I don’t come to a negative conclusion for everything I evaluate, but I wouldn’t want to be seen has endorsing weak or preliminary findings.

I’m considering creating a broad-based premium service. For a modest fee, subscribers would get weekly assessments of what I find in the scientific literature and how it is covered in social media. Subscribers could not only obtain this regular premium content, but also have the opportunity to nominate studies for review, with a subset selected and reported to all subscribers.

I know that my fans and followers have become accustomed to my providing a blog post and a social media presence without charge. I intensified this activity when I had a Carnegie Centenary Visiting Professorship at Stirling University, Scotland. One of my responsibilities was social engagement, and my involvement in blogging and social provided an effective and satisfying way of doing that. But now the professorship is over, and I have to think of alternative ways of monetizing my activities. I welcome readers’ thoughts and encourage them to sign up at CoyneoftheRealm.com to keep up on all of my activities.

 

My peer review of a PACE investigators’ article that the authors refused to heed

 

UPDATED 7/18/2017: OK, Michael Sharpe, I should not make fun of a serious matter. Being an investigator on the PACE trial has attracted you a lot of ridicule and cruel jokes.

I get it that 400 peer reviewed publications don’t qualify me as a reviewer of your paper, I am just not seasoned enough. But  could you maybe show me what you look for in a reviewer worthy of evaluating your manuscript?

mooning elf

“I am sure you could find many reviewers who are more qualified and who would do this. Maybe, you should advertise in social media.”

 

 

Chronic fatigue syndrome or myalgic encephalomyelitis (CFS/ME for short) is one of these.  The particular issue is the role of psychiatric or psychological approaches in the treatment of such patients.  Protest against this form of enquiry has been present for decades.  However, the increasing use of social media and blogs have co-ordinated and expanded the protest to an international one.  

-Professor Michael Sharpe

Yes, Professor Sharpe, Viva los blogs!  Viva los internationales! Your group has controlled  peer review. for too long and you are losing your grip.

hatfield mccoyAuthors do themselves a disservice by refusing to make changes suggested by reviewers, even when they have the power to do so.

I offered a tough-minded review of Peter White, Trudie Chalder, and Michael Sharpe’s Response to the editorial by Dr Geraghty   that recently appeared in Journal of Health Psychology with almost no responsiveness to my critique. The authors offered a partial correction of the authors’ misrepresentation of CONSORT as a guideline for conducting a clinical trial, rather than a checklist for reporting.

I don’t think that there was any ambiguity as to my identity or that I would be less than enthusiastic about what seemed to the authors’ hastily written response to Keith Geraghty.

Presumably Editor David Marks was of the same opinion as me about this general principle: Reviewers who have publicly disagreed with authors can often nonetheless offer valuable feedback, especially when editors interpret their reviews with their pre-existing opinion in mind – and the reviewer’s expertise.

I present my review below. I invite comparison to the published paper. I also invite the authors to share their bullying response, in which they threatened to get COPE involved. As empty and silly as a threat as what we have had to become accustomed to from Donald Trump. Make my day, Peter, Trudie, and Michael, proceed with  a complaint to the Committee on Publication Ethics (COPE).

The authors deserve to have an opportunity to respond to criticism of their work. However, this should not become an occasion for their repeating themselves to the point of self plagiarism, their invoking of dubious sources, or for their ignoring past criticisms.

I think there is a broader issue here – the effect of their trial and the way they have conducted themselves on the overall untrustworthiness of psychology, and health psychology interventions in particular. They flagrantly disregard basic standards for conducting and interpreting a randomized clinical trial, switch outcomes providing a dramatic demonstration of P hacking, attempt to block publication of criticism, and refuse to share their data, even when they have published with the requirement that it be made available. Particularly in this journal, it is important that these matters not be ignored. For the authors’ comment to be published in its present form, it would invite associating the journal with endorsement of untrustworthy psychological science.

For instance, the authors make a spirited defense of their right to switch outcomes. But they don’t address the good reasons that so many psychologists are fighting against such P-hacking, or for that matter, anyone concerned with the integrity of clinical trials. They are essentially arguing that they should have an exception, without acknowledging the important reasons why they are typically not granted. Should they be given a free pass to ignore efforts to reform both psychology and clinical trials?

The authors’ opening point about adherence to CONSORT displays an embarrassing ignorance of CONSORT. Surely they jest. The checklist is concerned not about the adequacy with which a trial is conducted, but the adequacy of the reporting of it. A randomized trial can be abysmal in its conduct, but possibly score close to perfect with the CONSORT checklist, if its flaws are appropriately transparently reported. The authors have made their point in numerous places and have been corrected. That they persist in making it reflects on the seriousness with which they approached responding to the paper which criticize them.

The authors point to patient involvement on the Trial Steering Committee and Data Monitoring and Ethics Committee. Anyone knowledgeable of international standards for these kinds of committees would find it astonishing that the authors/investigators themselves were involved on these committees particularly the data monitoring and ethics committee. Having been involved in numerous such committees, I think their presence would have the perception of compromising the requisite independent judgment of the committee. Patient advocates could legitimately question whether patients on the committee were representative or free from coercion from the investigators.

The short time in which one of the key papers was under review at The Lancet raises serious questions about the adequacy of its peer review. Moreover, the refusal by the editor of Psychological Medicine to consider a letter to the editor based on re-analyses using their originally selected primary outcomes raises issues about the integrity of peer review, both prepublication and post publication.

It’s extraordinary that 16 papers would come out of a single psychotherapy trial. The authors seem to be pointing to that as an accomplishment, but for others who are concerned with broader issues, it raises issues about duplicate publication and the integrity of the peer review under which those 16 papers were published.

I of course assume the Journal of Health Psychology is intended to reach an international audience and be responsive to international standards. I don’t think the author should be allowed to ignore the US committees which operated under considerable orderly and transparent rules in rejecting their diagnostic criteria and their assertions about the effectiveness of the treatment.

Their citing of  Action for ME (2011) is inappropriate for a number of reasons. It’s an unrepresentative that was not subject to peer review. The claims that patients believe their treatments lead to improvement of contradicted by the extraordinary petitions signed by thousands.

I’m not sure that citing of Action for ME (2011) is appropriate in this context, but the authors make a self-serving lumping of response categories and inaccurately portray the survey, Action for ME (2011). Given that Graded Exercise Therapy was also assessed the figures for it should also be given: 31% had received GET and 48% thought that it should be made available.

These are the kind of issues that would be missed by a casual reader, even a very intelligent and methodologically sophisticated reader who is simply not familiar with this literature and the authors as partisans for particular perspective on it that is not shared by all.

Advocates for the improvement of the trustworthiness of the psychological literature should be particularly offended by the distorted view offered in their point (3). They are dodging the very important issue of not only investigator allegiance, but investigator conflict of interest. Raising this issue about other interventions in other contexts have led to dozens of corrections and erratum. (I think it’s important that if this reply is published in any form, it should be accompanied by an editor-approved declaration of conflict of interest).

I’m quite familiar with the authors’ 2015 Lancet Psychiatry paper, but it is unrecognizable in the way is described in this comment. Numerous reasons why the authors cannot interpret the follow-up data in the way they do here and elsewhere are presented in letters to the editor, which the authors ignore. This is not the place to elaborate, but basically the authors abandoned their protocol and a substantial portion of the patients available for the follow-up were no longer receiving the treatment to which they were assigned.

The authors substantially misrepresent the use of Freedom of Information Act requests and their response to them. They exaggerate the number of requests that were made by counting multiple times any request that involved multiple variables. Furthermore, they misrepresent their responsiveness. They have released their data when they were involved as authors and had control over subsequent publications. They misrepresent the multiple times they’ve invoked the excuse that people who requested the data were vexatious.

I find it odd that they retreat to a blog post by Simon Wessely as a defense of their many methodological problems. The blog post was not peer reviewed and received a huge amount of substantive criticism. At best, Simon Wessely invokes his authority in place of evidence. Surely the authors can do better than an authority-based argument. It is my opinion that they should not embarrass themselves by bringing it the blog post here. If nothing else, I think they should respect the journal as a more formal forum.

Their excuse that they do not release their data because the consent forms do not allow it was argued in proceedings that cost them over 250,000 pounds. The final decision of the lower tribunal soundly rejected this excuse after reviewing it in explicit detail, including direct testimony from the author group. Here, as elsewhere in their reply, they are pleading for an exceptionalism which I cannot understand the basis.

It is debatable that improvement rates of 20% and 21% over 10% for the SMC alone group justifies a claim that the therapies “moderately improve” outcomes. But the authors do not exposed readers to this issue, just gloss over it.

I could continue with some serious substantive methodological, statistical, and interpretive issues. However, I think I have sufficiently established that the authors have not made effective use of their opportunity to reply to the editorial. Any effort to continue to exercise that option would have to be with a thoroughly revised manuscript requiring another peer review. What we see in the present version, however, is a thorough rejection of international standards, as well as the principled reasons behind efforts to improve the conduct and reporting of psychosocial interventions in clinical and health psychology.

There is another possibility available to the journal, however. Simply published the authors’ response as is, but allow reviewers to respond in print and to point out that the authors have repeated themselves, citing instances of this, and that the authors resisted encouragement to revise the manuscript from its present form. Rather than leaving this all behind the curtains, early investigators could get an interesting look into the process of challenging bad science and efforts to resist these challenges.

 

Dueling neurononsense: mindfulness training and gray matter

Confronting claims about mindfulness training changing the brain with claims that exposure to parks changes the brain suggests that neither claim had evidence.

Exhibit A

Mindfulness meditation training changes brain structure in eight weeks

mindfulness grey matter

participating in mindfulness.PNG

“Although the practice of meditation is associated with a sense of peacefulness and physical relaxation, practitioners have long claimed that meditation also provides cognitive and psychological benefits that persist throughout the day,” says Sara Lazar, PhD, of the MGH Psychiatric Neuroimaging Research Program, the study’s senior author. “This study demonstrates that changes in brain structure may underlie some of these reported improvements and that people are not just feeling better because they are spending time relaxing.”

What was the evidence for  these claims?

For the current study, MR images were taken of the brain structure of 16 study participants two weeks before and after they took part in the 8-week Mindfulness-Based Stress Reduction (MBSR) Program at the University of Massachusetts Center for Mindfulness. In addition to weekly meetings that included practice of mindfulness meditation — which focuses on nonjudgmental awareness of sensations, feelings and state of mind — participants received audio recordings for guided meditation practice and were asked to keep track of how much time they practiced each day. A set of MR brain images were also taken of a control group of non-meditators over a similar time interval.

How were these differences interpreted?

“It is fascinating to see the brain’s plasticity and that, by practicing meditation, we can play an active role in changing the brain and can increase our well-being and quality of life.” says Britta Hölzel, PhD, first author of the paper and a research fellow at MGH and Giessen University in Germany. “Other studies in different patient populations have shown that meditation can make significant improvements in a variety of symptoms, and we are now investigating the underlying mechanisms in the brain that facilitate this change.”

Amishi Jha, PhD, a University of Miami neuroscientist who investigates mindfulness-training’s effects on individuals in high-stress situations, says, “These results shed light on the mechanisms of action of mindfulness-based training. They demonstrate that the first-person experience of stress can not only be reduced with an 8-week mindfulness training program but that this experiential change corresponds with structural changes in the amygdala, a finding that opens doors to many possibilities for further research on MBSR’s potential to protect against stress-related disorders, such as post-traumatic stress disorder.” Jha was not one of the study investigators.

In general, it is not reasonable to make a big deal of a study comparing only 16 participants practicing mindfulness to 16 people who did not. Such a study is much too small to yield reliable, replicable differences. To be statistically significant in such a small study, differences between the two groups would have to be quite large. Mostly, large differences between groups found in small studies are exaggerated.

Even if moderate differences in grey matter were  associated with an intervention, the likelihood of them being shown in such a small study is considerably less than 50%. Meaning? That  these are most likelyspurious findings, even if real effects actually existed.

Even if such differences could be shown (and they were not reliably shown in this study, we wouldn’t know if anything specific about mindfulness caused them.

To make a fuss about these differences in terms of the function of these regions of the brain is silly.

Exhibit B: And now for the contrasting study.

Mental health: Parks and urban spaces can protect against mental decline in pensioners

parks protect

pic parks protect

Being surrounded by greenery was good for grey matter, as it was calming and quieter, according to the first study to look at the impact of parks on the mental health of pensioners.

Walking between busy urban environments and green spaces triggers changes in levels of excitement and engagement and reduces frustration.

The findings have important implications for architects, planners and health professionals with gardens and parks coming under pressure from austerity as we deal with an ageing population.

What was the evidence for these claims?

As part of the experiment, eight volunteers aged 65 and over from a wider sample of 95 wore an EEG (electroencephalogram) headset which recorded their brain activity as they walked.

The researchers also ran a video of the routes, asking the participants to describe ‘snapshots’ of how they felt. The volunteers were also interviewed before and after.

The study published in the International Journal of Environmental Research and Public Health found as well as preferring green space, it was beneficial to the volunteers’ brain health by being calming and quieter.

Co-author Dr Sara Tilley, of the University of Edinburgh, added: “To help ensure living longer is a positive experience for everyone, we need evidence based solutions to support lifelong health and well being.

Well, maybe, but this is not the kind of evidence that is needed. Numbers from such a small study are not evidence. But was the interpretation that was given?

“These findings – and others from the same project which show how important places are for our personal and cultural memories, and for enabling us to stay connected socially – have implications for the way we design for people of all ages, backgrounds and abilities so going outdoors in younger years becomes a lifelong passion for getting out and about.”

They mirror the findings of the 2013 study of younger adults from Heriot-Watt University, Edinburgh, which also found being surrounded by green reduced frustration and increased engagement.

Those researchers who also tracked brainwaves said parks affect the mind in a similar way to meditation.

I do not think either news item provides much evidence. We shouldn’t ignore the lack of sound evidence in comparing the two, but if we do, we have to say whatever is going on, is probably not very specific. But I doubt we have evidence that anything is going on that we did not have before these studies were introduced.

By the way, the parks-affect-the brain news item got more nonsensical if we had kept going:

A 2015 study of about 2,000 seven to 10 year old primary schoolchildren in Barcelona showed they improved memory and reduced inattentiveness in schools.

The findings may partly be explained by reduced exposure to traffic pollution, experts believe.

Other influences could include the psychological effect of having views of fields and trees rather than roads and buildings.

I will soon be offering e-books providing skeptical looks at mindfulness and positive psychologyeBook_Mindfulness_345x550, as well as scientific writing courses on the web, as I have been doing face-to-face for almost a decade.

Sign up at my new website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites.  Lots to see at CoyneoftheRealm.com.

What can we learn from an expensive but futile trial of interventions to improve child mental health?

My recent post at PLOS blogs Mind the Brain recent post at PLOS blogs Mind the Brain  concerned an Expression of Interest from the UK  Department for Education (DfE) for a large randomized trial comparing three preventive mental health interventions for promoting well-being among primary school children.

My first reaction was why is there no outcry from evidence-oriented mental health services researchers and policymakers against this waste of scarce resources? Is it no independent critical voice in the UK? Is this not a time for public engagement?

If we cannot depend on professionals to bring their expertise into the discussion, how about informed citizen-scientists parents who demand evidence for programs to which their children will be exposed and further evidence that the resources are not being squandered?

After all, the expensive initiative is being implemented when there is serious underfunding of both primary schools and child mental health programs. Many children with pressing mental health needs are not in a situation to obtain receiving treatment, and those who do obtain services are after an intolerably long wait. There are no prospects for substantial increases in funding in the future.

Maybe I don’t understand British understatement, but press coverage in the leading UK media outlets involved only the most muted of criticism and a lot of praise from those who will financially benefit from the opportunities that come with implementing and evaluating these interventions and the associated training that will be needed.

Some of the claims of the enormous potential benefit of these interventions were simply ludicrous.

Clearly there is an emphasis on expertise conferred by self proclamation or social status, not by demonstrated commitment to evidence. I recognized one of the individuals who wascited as an authority as someone who has repeatedly pointed to involvement in video games as responsible for the decline of Western civilization.

I detect a note of cynicism in calling for interventions that are delivered “lite,” with small amounts of time for week in which they are delivered by teachers who have been provided with minimal training.

I also find it cynical that the outcomes include not only self-report surveys, but improvements in classroom behavior and academic achievement. The schools and their teachers are set up to fail and to be blamed for not achieving these outcomes in the face of substantial underfunding.

I can envision a further initiative targeting the poor performance of teachers failing to meet important educational goals.

The three trialed interventions – mindfulness, protective behaviours, and breathing and relaxation training – lack sufficient evidence that they can improve these “harder” outcomes when delivered in the full form, much less when delivered “lite.”

There is a otential for harm to existing delivery of services from these programs. They create the illusion that something is being done. They tie up teacher time, which could be better devoted to  problems  that already highly visible in their classrooms.

In particular, protective behaviour involves encouraging students to complain of bullying or abuse but does not provide adequate ways of dealing with these problems. A lot of time will be spent dealing with intractable problems that are already known or the inevitable large number of false positives. Talking about bullying or abuse in a way that does not elicit effective response, exposes children to further bullying or abuse in retaliation.

There is no comparison/control group, and so little likelihood that the trial can conclude anything but no differences among the three interventions, but with the ambiguity that we can’t tell whether they are equally effective or ineffective.

Maybe there is a further cynicism among a small interconnected elite in a small country – let’s not to take a critical stance toward such wasteful and ineffective programs, lest we risk benefiting from future such initiatives by being identified as too critical of funded programs. Labeled as “too critical” reduces the likelihood of getting funded, but also a participate in the evaluation of future proposals because of a critical “bias.”

Along with my British colleagues, I am often called upon to evaluate grants for the German funding agencies, which are concerned about loyalties and conflicts of interest among German investigators interfering with dispassionate review of German proposals to improve physical health and mental health service delivery. Perhaps the British should consider similarly relying more on outside expertise, but I think there are cultural barriers to that happening.

I blog at a number of different sites including Quick Thoughts, PLOS blog Mind the Brain, and occasionally Science-Based Medicine. To keep up on my writing and speaking engagements and to get advance notice of e-books and web-based courses, please sign up at CoyneoftheRealm.com