Systematic review shows no improvement in quality of mindfulness research in 16 years

Should we still take claims about mental health benefits of mindfulness with a grain of  salt? A systematic review by one of mindfulness training’s key promoters suggests maybe so.

saltCritics have been identifying the same weaknesses in mindfulness research for almost two decades. This review suggests little improvement in 16 years the quality of randomized trials for mental health problems.

This study examined 171 articles reporting RCTs for:

(a) active control conditions, (b) larger sample sizes, (c) longer follow-up assessment, (d) treatment fidelity assessment, (e) reporting of instructor training, (f) reporting of ITT samples.

What was missed

Whether articles reporting RCTs had appropriate disclosure of financial or other conflicts of interest. COI pose significant risk of bias, especially when they are not reported.

This article discloses authors’ interests. One of the authors, Richard Davidson is a prominent promoter of mindfulness training.  A Web of Science search of Davidson RJ and mindfulness yielded 26 articles from 2002 to 2016. It would be interesting to check in see if these consistent weaknesses in mindfulness research are mentioned in these articles. To what extent do RCTs with Davidson as an author had these weaknesses, like being underpowered?

Critic: You say financial interests or other investments in a treatment are a risk of bias. Yet, this article is critical of mindfulness research. Wouldn’t you expect a more positive appraisal of the literature because of the authors having a confirmation bias?

Not necessarily. Conflicts of interest are a risk of bias, but don’t discredit an author, They only alert readers to be skeptical. Furthermore, the weaknesses in this literature are so pervasive, it would be difficult to put a positive spin on them.  Besides calling attention to specific weaknesses that need to be addressed in future research can become part of a pitch for more research.

The article

Goldberg SB, Tucker RP, Greene PA, Simpson TL, Kearney DJ, Davidson RJ. Is mindfulness research methodology improving over time? A systematic review. PLOS One. 2017 Oct 31;12(10):e0187298.

End of paper conclusion:

In conclusion, the 16 years of mindfulness research reviewed here provided modest evidence that the quality of research is improving over time. There may be various explanations for this (e.g., an increasing number of novel mindfulness-based interventions being first tested in less rigorous designs; the undue influence of early, high-quality studies). However, it is our hope that demonstrating this fact empirically will encourage future researchers to work towards the recommendations here and ultimately towards a clearer and scientifically-informed understanding of the potential and limitations of these treatments.

From the abstract

Objectives

The current systematic review examined the extent to which mindfulness research demonstrated increased rigor over the past 16 years regarding six methodological features that have been highlighted as areas for improvement. These feature included using active control conditions, larger sample sizes, longer follow-up assessment, treatment fidelity assessment, and reporting of instructor training and intent-to-treat (ITT) analyses.

Data sources

We searched PubMed, PsychInfo, Scopus, and Web of Science in addition to a publically available repository of mindfulness studies.

Study eligibility criteria

Randomized clinical trials of mindfulness-based interventions for samples with a clinical disorder or elevated symptoms of a clinical disorder listed on the American Psychological Association’s list of disorders with recognized evidence-based treatment.

Study appraisal and synthesis methods

Independent raters screened 9,067 titles and abstracts, with 303 full text reviews. Of these, 171 were included, representing 142 non-overlapping samples.

Results

Across the 142 studies published between 2000 and 2016, there was no evidence for increases in any study quality indicator, although changes were generally in the direction of improved quality. When restricting the sample to those conducted in Europe and North America (continents with the longest history of scientific research in this area), an increase in reporting of ITT analyses was found. When excluding an early, high-quality study, improvements were seen in sample size, treatment fidelity assessment, and reporting of ITT analyses.

Conclusions and implications of key findings

Taken together, the findings suggest modest adoption of the recommendations for methodological improvement voiced repeatedly in the literature. Possible explanations for this and implications for interpreting this body of research and conducting future studies are discussed.

Competing interests

RD is the founder, president, and serves on the board of directors for the non-profit organization, Healthy Minds Innovations, Inc. In addition, RD serves on the board of directors for the Mind and Life Institute. This does not alter our adherence to PLOS ONE policies on sharing data and materials

The variables examined in the systematic review

Six methodological features that have been recommended in criticisms of mindfulness research [10–12. 14]. These include: (a) active control conditions, (b) larger sample sizes, (c) longer follow-up assessment, (d) treatment fidelity assessment, (e) reporting of instructor training, (f) reporting of ITT samples.

…We graded the strength of the control condition on a five-tier system. We defined specific active control conditions as comparison groups that were intended to be therapeutic [17]. More rigorous control groups are important as they can provide a test of the unique or added benefit a mindfulness intervention may offer, beyond non-specific benefits associated with the placebo effect, researcher attention, or demand characteristics [11,14]. Larger sample sizes are important as they increase the reliability of reported effects and increase statistical power [11]. Longer follow-up is important for assessing the degree to which treatment effects are maintained beyond the completion of the intervention [10]. Treatment fidelity assessment allows an examination of the degree to which the given treatment was delivered as intended [12]. Treatment fidelity is commonly assessed through video or audio recordings of sessions that are coded and/or reviewed by treatment experts [18]. We coded all references to treatment fidelity assessment (e.g., sessions were recorded and reviewed, a checklist measuring adherence to specific treatment elements was completed). Relatedly, reporting of instructor training increases the likelihood that the treatment that was delivered by qualified individuals [12], which should, in theory, influence the quality of the treatment provided. Lastly, the reporting of ITT analyses involves including individuals who may have dropped out of the study and/or did not complete their assigned intervention [12]. Generally speaking, ITT analyses are viewed to be more conservative estimates of treatment effects [19,20], and are preferred for this reason.

 

Mindfulness: A web-based masterclass

eBook_Mindfulness_345x550

 

Like many of my loyal followers, I expected my e-book CoyneoftheRealm Takes a Skeptical Look at Mindfulness to be available by now. Though I’m disappointed that it is not, I’m pleased with the progress that has been made and how I have redefined the structure of the e-book.

My progress

Signs of the direction I am now taking with the e-book can be seen in recent blog posts at Mind the Brain and Quick Thoughts, but also in popular talks like the one I gave at this summer’s European Health Psychology Conference in Padova, Italy.

Coyne of the Realm Takes a Skeptical Look at Mindfulness  traces the claims being made about the practice of mindfulness back into the scientific literature in an inimitable style, wittingly challenging conclusions of bad research, but ultimately leaving readers with strategies for judging mindfulness for themselves.

The revised structure now involves discussion of some key papers in detail and provides readers with additional papers they can analyze themselves, with or without the tips I provide.

Feedback I have received suggests this format is a much improved way of providing tools for deciding for one’s self what, and how to think about mindfulness.  The outpouring of new mindfulness papers with even more outrageous claims about changing lives by changing brains continues. If you believe the social media, mindfulness is getting quicker and more powerful. Now you will be prepared for the pitch being made by promoters and be able to take apart what is being claimed and with what evidence.

Sample Chapters

Some basics of evaluating evidence

5. What are the best forms of evidence?

6. Observational studies: correlation does not equal causality

7. Evaluating comparisons of treatments

8. Evaluating reviews

9. Who says so? Importance of promoter bias and conflict of interest

Better than what? Evaluating comparisons between mindfulness and its alternatives

10. Typical clinical trials evaluating mindfulness: Some good, mostly bad

11. Head to head plus: Unusually informative mindfulness trials

12. Some clinical trials for you to evaluate (with hints)

To express my appreciation to those who preorder the book early, I would like to offer a special invitation as well as a free gift. By year’s end, my 100 minute video Master Class on mindfulness will be available for purchase. Although it will sell for $100.00, anyone who has purchased the CoyneoftheRealm Takes a Skeptical Look at Mindfulness book by November 10 will be receiving an email containing a code for a free Mindfulness Master Class video.

In addition, if you preorder the e-book CoyneoftheRealm Takes a Skeptical Look at Mindfulness before November 10, you will  have an opportunity to submit questions to a pool from which those will be drawn to be addresse in a Q&A session at the end of the video.

So,  preorder the mindfulness e-book or the full set of three Coyne of the Realm Takes a Skeptical Look at… and then submit your nominations for questions to be asked in the Q&A session to james@coyneoftherealm.com.

 

How advice gurus sell more products when corporations discover mindfulness training doesn’t work

competative advantage

Corporations purchasing expensive mindfulness training packages for corporate leadership and rank and file employees inevitably discover they do not obtain the benefits that are claimed for mindfulness. How can this become a strategic opportunity for advice gurus to sell more products?

value of happinessIn a recent article in Harvard Review of Business (HBR) article, Daniel Goleman and Matt Lippincott deftly explain that if corporations still believe in the promises of mindfulness, they should purchase their product, training in emotional intelligence. They claim the support of unpublished research, but, as we will see, the published research casts doubts on their product being backed by much evidence.

book cover_24582677-2ecc-4456-a2c1-c2eadfcb8d5b.jpgNo matter. We have a fascinating example of associations with Harvard, research, and University of Pennsylvania being used to brand an advice product as effective and backed by science.  It is a strategy that Amy Cuddy used to present herself as an advice guru before the launch of power posing. Elizabeth Dunn and Michael Norton use the HRB to launch their campaign for the science of smarter spending, how money can buy you happiness, if you follow their advice.

The click link “What really makes mindfulness works” suggests an article from Harvard Review of Business explainswhat really makes mindfulness training work. Actrually, the article actually pitches products for when corporations find mindfulness does not  live up to expectations.

Without Emotional Intelligence, Mindfulness Doesn’t Work

The first two paragraphs skillfully criticize mindfulness as a fad, but argue for a “complicated relationship” between mindfulness and improved executive performance, setting the reader up for a pitch for their product, which provides a more effective route.

Mindfulness has become the corporate fad du jour, a practice widely touted as a fast-track to better leadership. But we suspect that not all the benefits laid at its feet actually belong there. Our research and analysis has revealed a complicated relationship between mindfulness and executive performance—one that is important for leaders to understand as they seek to develop in their careers.

 Mindfulness is a method of shifting your attention inward to observe your thoughts, feelings, and actions without interpretation or judgment. A mindfulness practice often begins simply by focusing on your breath, noticing when your mind wanders, and then bringing it back to your breath. As you strengthen your ability to concentrate, you can then shift to simply noting your inner experience without getting lost in it at any point in your day. The benefits attributed to this kind of practice range from stronger relationships with others to higher levels of leadership performance.

The next paragraph introduces Sean, “a senior leader at a Fortune 100 corporation” who will tell you that mindfulness played a critical role in transforming his career.

To allay readers’ suspicions that Sean may be a fiction contrived by the authors to make a point, they next claim he is one of  “42 senior leaders from organizations throughout the world who practice mindfulness and whom one of us (Matt Lippincott) studied at the University of Pennsylvania.”

This “research” is described in HBR as producing a promoter’s dream list of benefits to practicing mindfulness. Unfortunately, no link is provided to an actual report of methods and results.

I Googled Matt Lippincott. A link to ResearchGate came up

Lippincott has no published research listed, but there was a link to an unpublished dissertation.

This qualitative research study examined detailed reports by senior organizational leaders linking mindfulness to improved leadership effectiveness. Extensive research supports the existence of a relationship between mindfulness and cognitive, physiological, and psychological benefits that may also have a positive impact on leadership effectiveness. Currently, however, little is known about the processes potentially enabling mindfulness to directly influence leadership effectiveness, and as a result this study was designed to explore this gap in the literature. Data was collected through in-depth interviews with forty-two organizational leaders in North and South America and Europe, many with a history of leadership roles at multiple global organizations. Participants credited mindfulness for contributing to enduring improvements to leadership capabilities, and data analysis revealed new findings clarifying the perceived relationship between mindfulness and tangible results for organizational leaders. Specifically, the results indicate that mindfulness is perceived to contribute to the development of behaviors and changes to awareness associated with improved leadership effectiveness. A potential relationship between mindfulness and the development of emotional intelligence competencies linked to increased leadership performance was revealed as well. The contribution of this study to current literature is also discussed, as are recommendations for future research.

So a dissertation with a weak methodology that allows invoking “research” and “University of Pennsylvania” for credibility.

Back to the wrap up of the HBR article, we get the buy-our-product punch line:

We believe that by focusing on mindfulness-as-corporate-fad, leaders run the risk of missing other opportunities to develop their critical emotional skills. Instead, executives would be better served by deliberately assessing and improving their full range of emotional intelligence capabilities. Some of that work may well involve mindfulness training and practice, but it can also include formal EQ assessment and coaching. Other tools and approaches include role-playing, modeling other leaders you admire, and rehearsing in your mind how you might handle emotional situations differently. By understanding that the mechanism behind mindfulness is the improvement of broader emotional intelligence competencies, leaders can more intentionally work on all of the areas that will have the strongest impact on their leadership.

The two authors

Daniel Goleman is Co-Director of the Consortium for Research on Emotional Intelligence in Organizations at Rutgers University, co-author of Primal Leadership: Leading with Emotional Intelligence, and author of The Brain and Emotional Intelligence: New Insights, Leadership: Selected Writings, and A Force For Good: The Dalai Lama’s Vision for Our World. His latest book is Altered Traits: Science Reveals How Meditation Changes Your Mind, Brain, and Body.

Matthew Lippincott is a business owner, researcher, and author involved in the creation of new leadership development solutions. He holds a doctoral degree from the University of Pennsylvania, and has previously held leadership positions at two of the world’s largest software companies.

We get a sense of former Psychology Today editor Daniel Goleman trying to create a brand of advice that unites the Dalai Lama, the good, and corporate competitiveness. Lippincott has delivered a Penn Wharton dissertation uniting mindfulness and emotional intelligence. He got work on this basis.

Hmm, did the link emerge from the data, or is this dissertation simply an informercial aimed at getting a job marketing the combo of mindfulness and emotional intelligence?

The scientific status of emotional intelligence

According to Wikipedia:

Emotional intelligence (EI) is the capability of individuals to recognize their own and other people’s emotions, discern between different feelings and label them appropriately, use emotional information to guide thinking and behavior, and manage and/or adjust emotions to adapt to environments or achieve one’s goal(s).[1]

Although the term first appeared in a 1964 paper by Michael Beldoch, it gained popularity in the 1995 book by that title, written by the author, psychologist, and science journalist Daniel Goleman. Since this time, Goleman’s 1995 analysis of EI has been criticized within the scientific community,[2] despite prolific reports of its usefulness in the popular press.

The Wikipedia offers three stinging critiques backed by links to references.

Emotional intelligence cannot be recognized as form of intelligence

Goleman’s early work has been criticized for assuming from the beginning that EI is a type of intelligence or cognitive ability. Eysenck (2000)[57] writes that Goleman’s description of EI contains unsubstantiated assumptions about intelligence in general, and that it even runs contrary to what researchers have come to expect when studying types of intelligence:

“[Goleman] exemplifies more clearly than most the fundamental absurdity of the tendency to class almost any type of behavior as an ‘intelligence’… If these five ‘abilities’ define ’emotional intelligence’, we would expect some evidence that they are highly correlated; Goleman admits that they might be quite uncorrelated, and in any case if we cannot measure them, how do we know they are related? So the whole theory is built on quicksand: there is no sound scientific basis.”

Emotional Intelligence confuses skills with moral qualities.

Adam Grant warned of the common but mistaken perception of EI as a desirable moral quality rather than a skill, Grant asserting that a well-developed EI is not only an instrumental tool for accomplishing goals, but has a dark side as a weapon for manipulating others by robbing them of their capacity to reason.

Emotional Intelligence has little predictive value.           

Landy (2005)[61] claimed that the few incremental validity studies conducted on EI have shown that it adds little or nothing to the explanation or prediction of some common outcomes (most notably academic and work success).

Some further links I also discovered

Steve Topak’s Don’t Believe the Hype Around ‘Emotional Intelligence’ 

What if I said that emotional intelligence is the ability to recognize, understand and control emotions – not just our own but the emotions of others, as well? What if I said it can be used to manipulate behavior? That sounds a bit different, doesn’t it? Not such a no-brainer anymore, is it?

And

This is not some sort of rhetoric slight of hand nor is that definition controversial. It’s common doctrine. But if authors, consultants and executive coaches were to say that Adolf Hitler was as adept at emotional intelligence as Martin Luther King Jr. – as Adam Grant explains in The Atlantic – they would not sell many books or book a lot of gigs.

Excerpts from a nice comprehensive review

Kilduff M, Chiaburu DS, Menges JI. Strategic use of emotional intelligence in organizational settings: Exploring the dark side. Research in organizational behavior. 2010 Dec 31;30:129-52.

Just as the cognitively smart person may be able to understand options and draw conclusions quickly and competently, so the emotionally intelligent person may be able to assess and control emotions to facilitate the accomplishment of various goals, including the one of getting ahead. We suggest that high-EI people (relative to those low on EI) are likely to benefit from several strategic behaviors in organizations including: focusing emotion detection on important others, disguising and expressing emotions for personal gain, using misattribution to stir and shape emotions, and controlling the flow of emotion-laden communication.

And

We have shown that the strategic disguise of one’s own emotions and the manipulation of others’ emotions for strategic ends are behaviors evident not only on Shakespeare’s stage but also in the offices and corridors where power and influence are traded.

Take away messages.

Don’t expect top quality science from Harvard Business Review, but sliced and diced stuff to sell products a research-based.

Be skeptical of researchers who promote their studies in HBR. They are often publishing an infomercial that to be effective must make extravagant claims that require stronger and more unambiguous findings than research  can possibly produce.

eBook_Mindfulness_345x550I will soon be offering e-books providing skeptical lookseBook_PositivePsychology_345x550 at mindfulness and positive psychology, and arming citizen scientists with critical thinking skills so they dan decide for themselves. I also be offering scientific writing courses on the web as I have been doing face-to-face for almost a decade. I want to give researchers the tools to get into the journals where their work will get the attention it deserves.

Sign up at my website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites. Get advance notice of forthcoming e-books and web courses. Lots to see at CoyneoftheRealm.com.

A longer than needed look at ’11 minutes of mindfulness can reduce drinking by bottle of wine a week.’

Why University College, London issued embargoed press release for an already published mindfulness study is more interesting than the insipid study itself.

The study is actually provides an excellent teaching example of what can happen when a recipe for hype-worthy mindfulness study is mindlessly followed.

University College, London should put more money into providing competent methodological and statistical consultations than orchestrating publicity campaigns promoting studies of mindfulness laced with pseudoscience.

A journalist sent me an embargoed press release from University College, London. At first I couldn’t make sense of why there was embargo, because the study itself was already available on the web.

embargoed for release

Apparently UCL was trying to coordinate a media campaign for a mediocre study by coordinating churnaling – essentially plagiarizing by journalists – in the media like a chorus of quacking ducks..

Not surprising, sources, like The London Times and Time complied fabulously.

time

How A Short Meditation Can Help People Drink Less

times mindfulness

https://www.thetimes.co.uk/article/mindfulness-can-help-you-to-drink-less-ljvvv5j6z?shareToken=31e1c14b70c37de9242bc765186e19f5

New sources churnaled the press release, which said:

Brief training in mindfulness strategies could help heavy drinkers start to cut back on alcohol consumption, finds a new UCL study.

After an 11-minute training session and encouragement to continue practising mindfulness — which involves focusing on what’s happening in the present moment — heavy drinkers drank less over the next week than people who were taught relaxation techniques, according to the study published in the International Journal of Neuropsychopharmacology.

“We found that a very brief, simple exercise in mindfulness can help drinkers cut back, and the benefits can be seen quite quickly,” said the study’s lead author, Dr Sunjeev Kamboj (UCL Clinical Psychopharmacology Unit).

And

“We used a highly controlled experimental design, to ensure that any benefits of mindfulness training were not likely explained by people believing it was a better treatment,” said co-author Dr Tom Freeman (Senior fellow of the Society for the Study of Addiction), who was part of the research team while based at UCL.

The mindfulness group drank 9.3 fewer units of alcohol (roughly equivalent to three pints of beer) in the following week compared to the week preceding the study, while there was no significant reduction in alcohol consumption among those who had learned relaxation techniques.

“Practising mindfulness can make a person more aware of their tendency to respond reflexively to urges. By being more aware of their cravings, we think the study participants were able to bring intention back into the equation, instead of automatically reaching for the drink when they feel a craving,” Dr Kamboj said.

The study itself was not very thoughtfully constructed according to a familiar recipe:

1. Appeal to neoliberal policies by setting out to claim that citizenry can learn to self-discipline themselves away from unruly behavior with cheap methods that will distract from underfunding of health and substance abuse services.

2. Take some speculations about how mindfulness applies eastern meditation techniques to change health behaviors and apply them to drinking alcohol.

3. Ignore existing conventional literature concerning skills relevant to avoiding excessive alcohol consumption.

4. Rely on effect sizes from an underpowered study of mindfulness for reducing smoking. Recruit exactly the same number of participants.

5. Assemble a battery of psychological measures that might conceivably be affected by mindfulness meditation or an imbalance of nonspecific effects between groups.

6. Assemble a set of biological measures claimed in the literature to be biomarkers for a state of contemplative mindfulness.

7. In response to criticism that past mindfulness studies have been unblinded comparisons of mindfulness to inert conditions, claim blinding of experimenters and participants and use an active treatment, relaxation.

8. Avoid measurement of whether participants actually practiced mindfulness.

9. Ignore indications from full study analyses whether within-group analyses would make for attractive results.

10. More generally, significance-chase and selectively report favorable results.

What can go wrong?

1. The authors could take advantage of lower standards of evidence required to promote results appealing to neoliberal prejudices. Yet they may have gotten carried away with claims only 11 minutes of mindfulness reduce drinking equivalent to a three pints of beer or a bottle of wine per week. And the adverse social consequences of these claims being accepted:

Weak designs with short-term subjective self-report outcomes proclaiming dramatic results can lead to continued underfunding of services and denial of more intensive services to those most in need.

2. Armchair speculations from the mindfulness literature are poor substitute for functional analysis of behavioral processes leading towards moderate versus excessive drinking. Really, let’s let some common sense in:

3. By the time that people are contemplating beer in their mouths, it may be too late for them to engage more adaptive and effective practices. Furthermore, concentrating too much on the taste of beer may be counterproductive for those who enjoy it and who would be better off focusing on distracting social cues. Namely, they could be concentrating on the conversation and the people around them drinking less, rather than getting focused on the beer in their mouth.

4. As expected, effect sizes from underpowered study in a different population and different clinical focus proved exaggerated. Moreover, given the study’s distinctive claim about elucidating the biological underpinnings of mindfulness, a much larger sample would’ve been needed to be geared to finding of facts in those outcomes if they were indeed present.

 

5. Many of the other psychological measures were poorly chosen and irrelevant to evaluating the intervention, but were nonetheless selectively presented. I won’t waste my readers time keeping score as I cut through this underbrush.

6. The assessment of particular biological correlates of mindfulness was a both a matter of convenience of measurement in a short session and belief in the myth that such correlates are not only biomarkers, but the mind-body indicators that participants had achieved a state of contemplative mindfulness in the process of listening only to 11 minute tape.

A biological correlates does not a biomarker make, and less specificity can be established and cross validated cut points for mindfulness verses-not mindfulness. What is also ignored is that there are no specific “biological markers” for mindfulness. Any correlates with mindfulness training are often due to nonspecific aspects of the treatment, if they are not noise.

Cardiac vagal tone is particularly dubious as a measure. It’s highly responsive to contextual factors and has large intra-and inter-individual variability. Moreover, it is susceptible to changes in breathing and so it is not surprising that changes were greater in the relaxation group. In general, it would take a much larger study under much more carefully controlled conditions to produce meaningful results.

The authors conveniently ignored that other than when conducted by promoters of transcendental meditation, reliable differences in blood pressure are typically not found. With such a grossly underpowered sample, any apparent differences are likely to be artifactual, due to outliers, or simply noise being interpreted as music..

7. The authors claim that a strength of their study is that they have a double-blinded comparison of two active treatments. Neither research assistants nor participants were told which was the mindfulness nor relaxation condition. Yet, the mindfulness tape gave much clearer instructions as to the experimenters intentions and expectations about how effects would be reported in subjective self-report measures. It’s not clear why participants would expect getting relaxed would reduce alcohol consumption.

The authors seem unaware of the literature concerning how easy it is to induce changes in subjective self-report of short-term alcohol consumption. In the 80s, a number of us did studies in which the active intervention was simply eliciting reports of alcohol consumption and giving participants feedback that it was more than peers rank or compared to some normative standard. I seem to recall that if you follow that literature back further, you’ll see that this active treatment had started as a control condition with significant effect on self-reported alcohol consumption in the short term. I’m very skeptical about whether such changes persist over the long-term, except in participants otherwise motivated to reduce the alcohol consumption.

8. Despite the assembly of a full range of psychological and biological measures, the authors fail to access whether participants actually went home and practice mindfulness. They don’t report any attrition, but that might be in effect a participants been able to report in remotely, rather than not showing up for reassessment as part of an otherwise noncompliance with the study.

9. The abstract of the published study clearly indicated the point at which the authors commit themselves to significance chasing and p-hacking.

Both groups showed acute reductions in craving after training, although a trend Group x Time interaction (p=0.056) suggested that this reduction was greater in the relaxation group (d = 0.722 P < .001) compared with the mindfulness group (d = 0.317, P = .004).

10. I could go on, but I think you get the picture. An overwhelming number of statistical tests within and between groups on a full range of often irrelevant variables are mustered to make the case that more of this sort of research is needed. The spin on the findings in the press campaign can only serve to encourage more of this research and strengthen the tendency towards confirmatory bias. But in the end, the authors hypothesized a group by time interaction in changes in cravings and did not obtain it, although admittedly in an underpowered study. But they made it into Time magazine and The London Times described in exactly the words they provided.

As an experiment, I did not provide my usual links to relevant literature. I’ve been disappointed in how few readers click through and actually access those links, despite my efforts to make many of them open access. However if any readers want to take issue with me, I invite them to provide links to studies at odds with what I’m saying and I will see what I can come up with in response.

Is acceptance and commitment therapy (ACT) in a post-evidence phase?

Steve Hayes encouraged me to give the evidence for the efficacy of ACT another look. I did and I wasn’t impressed.

Recent interviews with three founders of ACT make little reference to evidence, but a lot of reaching for roles as gurus, not as trainers in evidence-based therapy.

I recently tweeted about acceptance and commitment therapy (ACT).

my tweet

I got a quick reply from Steve Hayes:

edited steve hayes exchange

I checked the document, and had lots of numbers, but little indication of accumulating quality studies. I recognized some of the meta-analyses were quite bad.

ACBS

The document listed 2 organizations that describe ACT, or areas of ACT, as evidence based, APA Division 12 and SAMHA. Both sourceds depend heavily on promoters of a particular approach assembling and evaluating material, with all the biases that that introduces, not independent, systematic evaluation. The APA page was written by an ACT his enthusiasts, but was nonetheless not particularly impressive. Only ACT for chronic pain received an evaluation of more than modest support.

American Psychological Association, Society of Clinical Psychology (Div. 12), Research Supported Psychological Treatments.

Chronic Pain – Strong Research Support

Depression – Modest Research Support

Mixed anxiety – Modest Research Support

Obsessive-Compulsive Disorder – Modest Research Support

Psychosis – Modest Research Support

I checked the most recent review and meta analysis of ACT for chronic pain.

Veehof MM, Trompetter HR, Bohlmeijer ET, Schreurs KM. Acceptance-and mindfulness-based interventions for the treatment of chronic pain: a meta-analytic review. Cognitive Behaviour Therapy. 2016 Jan 2;45(1):5-31.

The review combined  28 studies evaluating mindfulness or ACT. It did not provide separate evaluations for the two treatments. There were 9 studies comparing ACT to routine care or wait list. Only one had more than 35 patients in the smallest group at follow up, a study of ACT with fibromyalgia patients. There was one study comparing  ACT with CBT having such a minimal sample size, and the two treatments did not differ. There were two comparisons with multidisciplinary treatment, but neither met this minimal sample size.

A systematic review that is almost as recent further showed what bad shape the literature concerning ACT for pain is in.

Hann, K. E. J. & McCracken, L. M. (2014). A systematic review of randomized controlled trials of Acceptance and Commitment Therapy for adults with chronic pain: Outcome domains, design quality, and efficacy. Journal of Contextual Behavioral Science, 3, 217-227.

The review discretely suggests that the weak designs and potential for p-hacking of the typical study of ACT for pain would allow even homeopathy to look better than nothing.

A systematic search identified 1034 articles and ten studies were selected as eligible for review. Overall, 15 outcome domains were assessed using 39 different measurement tools across the ten RCTs. The outcome domains assessed in the reviewed trials were, to an extent, in-line with recognized guidelines. Six of the ten studies identified primary and secondary outcomes; one included just one outcome and three did not categorize outcomes. All ten trials included a measure of some aspect of psychological flexibility; however these were not always formally identified as process variables. Pain and emotional functioning were the most frequently measured outcome domains. A review of outcome results suggests that ACT is efficacious particularly for enhancing general, mostly physical functioning, and for decreasing distress, in comparison to inactive treatment comparisons. It is recommended that future RCTs (a) formally define outcomes as primary, secondary and process variables, (b) consider including measures of physical or social functioning, rather than pain and emotional functioning, as primary outcomes, (c) address existing risks of bias, such as reporting bias, and (d) include more components of psychological flexibility, such as cognitive defusion and self-related variables.

Are the leaders of the ACT movement giving up on accumulating evidence and reaching for guru status? Maybe, if some recent interviews are any indication.

3 Founders of ACT in BPS Psychologist

Aseries of interviews by Kal Kseib with three founders of ACT can be found in the BPS Psychologist– Kelly Wilson, Kirk Strosahl and Steve Hayes.

Kelly Wilson

The title of the interview with Kelly Wilson suggests he is going to pluck some flowers stuck between his toes in his Gucchi sandals 

Kelly you find some connection

Kelly doesn’t disappoint.  He does mention evidence three times. Twice here:

There’s interesting evidence out there about how even just small doses of such things can make a difference. In research looking at sedentary behaviour, just a couple of minutes each hour of getting up and moving around is shown to disrupt some of those destructive metabolic processes. Reducing exposure to toxins including social toxins, moving your body, getting enough sleep, eating nutritious food, engaging in some small mindfulness practice – every bit of evidence seems to suggest that really small amounts matter. And if you look at behavioural activation, which is one of the tier 1 treatments for depression, it’s about ‘get them started’. The smallest thing – even if just your feet move! Go down to the corner, go out for coffee, go out to a movie with a friend. That work was really built on the idea that these patterns of small activities give a chance for the stream of life to kind of pull you back in.

And then

For instance, there’s evidence to show that people with high levels of self-stigma about their obesity massively increase their risk of all-cause mortality and cardiac mortality.

I wouldn’t interpret modest effect sizes in observational studies in this way. But whatever….

Then there is the too-much-information, grab-your-guru-status disclosure.

What’s something people might not know about you that, if they knew, would surprise them?

I came to psychology late. I was 30-years-old when I started college. I dropped out of school when I was 16. And between the ages of 16 and the age of 30, I was a drug addict and an alcoholic. Chronically unemployed and unemployable – a serial felon. Terribly, terribly depressed – suicidally depressed pretty much all the time, except for little glorious windows in time when I’d get just the right combination of drugs and I’d be able to feel like I could stay in my own skin for a minute. But mostly it was just hard and destructive to me and to a lot of other people. In 1985, at the age of 30, after many years of overdoses and car accidents and violence, illness, and just wanting to die, I was admitted into a locked psychiatric ward. I joke with people sometimes – although it’s true – that I got my start in psychology in a psychiatric hospital

I have written about aspiring TED talk gurus like Amy Cuddy and Kelly having to find their redemptive selves, linking the product they sell to overcoming the adversity in their lives. I think there may be some literary license in play here, but if it is meant to inform, rather than just entertain, what is the takeaway message?

What is the greatest lesson life has taught you recently?

That the things you love the most are also the things that are the sources of the most extraordinary pain possible. And there is no greater joy or pain life can deliver than that you get from your children. Sweet and sad, poured from the same vessel in equal measure.

What motivates you to inspire self-care in others?

I lost my eldest brother Randy in 1987 to suicide. Since then, I lost my brother David in 2011 to a cerebral haemorrhage, and my baby brother Michael in 2013, to a heart attack. In 1998 I was given a head and neck cancer diagnosis. My interest in self-care, kindness and wellbeing is not a casual interest, it has to do with me, it has to do with the people who I love, both the ones who I know, the ones I’ve lost, and the ones who I don’t know yet.

Kirk Strosahl

Maybe he is just being flippant, but I was uncomfortable with the strong, but implicit suggestion of Kirk’s title that people chose to suffer.

Kirk pain is inevitable

How would you describe ACT in a nutshell?

I think ACT tries to promote people who are open to their own experience and can separate themselves from the literal meaning of their experience – so that they’re not governed by thoughts or feelings or emotional reactions to things. They are in touch with their personal values so that they are geared toward living life to its fullest.

What are your most inspired actions?

I think the time I truly felt the most inspired simply in terms of output was writing my most recent book, Inside This Moment. I was in a zone for the entire time that I was writing it, it was coming from the heart. It was actually quite an amazing experience. The book zeroes in on how to use the present moment in therapy to inspire people to live their lives to the fullest, and not to run from their own demons, but rather cradle them. So that was enjoyable. Then I’ve also always been inspired to work with Patti [Dr Patricia Robinson]. We have a lot of fun and there’s intellectual growth when we write and get our arms around stuff together. So I’ve been very lucky that way.

Where are ACT and the other so-called ‘third-wave therapies’ heading now?
I think people are going to have to get their arms around what we mean by ‘mindfulness’ in a much more scientifically sound sense. Not the term, not the popular concept, but understanding exactly what goes into it. We’re still very overly general about it in our conversations, and because of that we’re losing leverage in therapy that we would otherwise gain by being more discriminating. That’s one area I think is going to see a lot of further growth. And I think we’re going to be looking at how we get our treatments briefer without losing effectiveness, because resource systems out there simply can’t afford the longer-term version of CBT or ACT for the masses – that’s another big area. How are we going to populate these concepts into public health models and work with lay people, primary care providers and teachers, for example? We’re still very therapist-centric in our profession, and because of that our population health effectiveness has been extremely limited.

I think the other area that is going to become bigger and more important is values-based behaviour change. It’s not like people haven’t been exploring it, but I think the amount of development will increase. There are measurement issues that are going to have to be dealt with, as well as creating more efficient ways of talking to people about motivating factors in their lives.

Kirk is modest in his aspirations for ACT, but I don’t see the relevance of evidence.

What is the greatest opportunity ACT has in today’s world?

I actually think that, if we don’t fall on our own sword, it’s going to be about bringing mindfulness concepts into the general public and Western civilisation. That’s not going to mean getting everybody in the West to put in hours of practice a day, sitting on a pillow ‘umm-ing’. Rather it’s this idea that these are actually pretty portable interventions based on neuroscience. They have a very rapid effect on brain neural pathway development and brain efficiency, and they don’t have to be these onerous, lifelong practices. There’s going to be a huge opportunity there if we can get the right message to people – that these are things you can teach yourself and your brain in small bits, and that it’s more about persistence than the amount of time you take. It’s about doing things intentionally and practising intention, as well as practising paying attention. These two things go hand in hand in mindfulness – this ability to pay attention in a particular way and then to act with intention inside of your own space. To me those things are so intricately linked to psychological health that if we could get those out into the public domain in ways which didn’t seem overwhelming to people, that would be a huge accomplishment for ACT, or for any of the mindfulness-based therapies that could get this figured out. So it isn’t just in the hands of a few peopl

Are ACT and CBT actually little more than saying to people ‘live with it, or change it’?
There are hidden properties of treatments, and then there are the observable properties of treatments. In ACT the observable properties are quite different, with an emphasis on values. It’s saying to people ‘don’t just tolerate your life, build your life from within’. ACT uses values as a foundation for addressing that.

Then there’s the ability to create space between you and what starts to show up in your life when you start doing things that matter. It’s an optimistic treatment that assumes people can do amazing things if they get lined up behind the right psychological processes. And it might well be that in CBT, even though ostensibly focusing on helping people change thoughts and behaviours, the act of talking about thoughts and behaviours is in a way itself a kind of a ‘defusion’ intervention. That’s what I mean by ‘hidden’ properties. You think that the mechanism is about the client becoming more logical and less irrational, but it may in fact be that by talking about thoughts you’re actually doing ‘defusion’ without even realising it.

I am also uncomfortable with the message that effective therapy can be delivered in 15 minutes. I am sure it sells well in behavioral health settings or in neoliberal societies where the goal is to underfund mental health services, but claim to offer them broadly. But strong claims with such potentially socially pernicious implications need very strong evidence. I have searched and can find none.

You’re a specialist in delivering brief, or ‘focused’ ACT interventions to patients – sometimes as brief as 15 minutes. Could you give a picture of what you might focus on in a session, say for diabetes or depression?
The goal of focused ACT is to get patients to make direct contact with the unworkable results of their current life strategies. Usually, these strategies involve avoiding dealing with important life issues – such as maintaining social health, managing diet or other health risk behaviours in the case of diabetes. The counter-weight in focused ACT is to get the patient to make direct contact with what matters to them in their life, and whether their avoidance behaviours are helping them move in that direction. This discrepancy creates a ‘healthy anxiety’, which we encourage patients to accept as a ‘signal’ that some type of change in personal strategy is needed. You don’t get people to change behaviours by giving them a label, or scaring them with adverse consequences if they don’t change, or lecturing them about the necessity of change. Change comes from within, not from without. Most patients know implicitly that they are avoiding things, but they don’t want to be condescended to, criticised or cajoled about it. So focused ACT is a very humanising approach in which we readily agree that making important changes in life might likely trigger painful emotional consequences or distressing memories of past failures, et cetera. Pain is inevitable, but suffering is optional.

The therapist and patient are on the same journey in this, and they’ve just happened to run into each other. There is no difference really between us.

Steve Hayes

Last, but certainly not least, Steve’s interview is titled with a rhetorical question that he does not get around to answering.

steve why can't we

How do your ideas connect with your values?

I’ve tried in my career to be less concerned about brand names and personal applause, and more concerned about building community and connecting to this larger value of ‘how can we best serve people using science?’. I get more excited in my personal work about supporting the work of others. I mean I’m productive, I do research, I write books. But to this day I’ve never not responded to an email. If a patient contacts me and says, ‘I’m in misery, what do I do?’: yeah, it might be two or three sentences, but I’m going to try and do something. So my value is being of use and building a community that can profoundly be of use – trying to bring science into alignment with what society needs and wants from us. And I don’t think we’ve always served that.

Surely, you jest, Steve.

You’re one of the founders of acceptance and commitment therapy, or ACT. What is the greatest opportunity ACT has in today’s world?

Reaching the culture. My son’s favourite cartoon show is Steven Universe – it’s very sweet. Garrett is a kind of a meditation or mindfulness person, a wise adult. She sings a song called ‘Here comes a thought’, and the show notes say she got it from ACT. The first stanza is ‘flexibility, love and trust’, and it walks through what to do with difficult thoughts. It talks about seeing and feeling, and watching the thought. And they actually lie on the ground like they’re watching clouds. I started crying seeing it because I’m thinking, ‘how many children have shame that they’re carrying?’ It’s got to be a large percentage of children, right? And they can’t talk to their parents, and they can’t talk to their peers, because what is it they’re going to say? ‘Just don’t think about it.’ And it’s like pouring gasoline on a fire, it’s just a horrible thing to do to shame… it’s saying there’s something wrong with you for feeling and thinking that, and you just need to think differently and it’ll go away. We can do something a lot wiser than that.

When I teach brand new students, say, research methods, the very first thing I get across is this: the modal number of citations for a publication in psychology is zero, which means nobody has been influenced by it enough to actually put in writing that it mattered. So I only want something that you really have heart for, something that deep down you think might really matter. And then I tell them, ‘and by the way, your research idea almost certainly is a bad idea’, because that’s how it [the modal number of citations] gets to zero. From there we can figure out a way that maintains the heart, and that won’t have zero citations. There’s like this dialectic of ‘how do we come into our field as social human beings with this core of caring, and nurture and sustain that, but also learn how to channel our interest and questions in a way that will have a long term impact on the world – on our clients and on our colleagues?’.

What’s something people don’t know about you, that if they knew, they would be surprised?
I think people are sometimes surprised that if I wasn’t going to be a psychologist, my other big thought about what I would be is a carpenter. I’m a veteran re-modeller, I’m tearing down my house all of the time. People sometimes think I’m a geek, but I can put in plumbing and I’ve built an entire house from nothing, from the foundation to the roof – done everything myself. So I’m a builder and I try to bring that sensitivity into what I do research-wise and clinically also. I want to build something that lasts and that matters, and I think we’ve been able to do it in some of the work we’ve been doing in CBT and ACT, and Mindfulness and ACBS etc.

I can’t add to this. I will just let Steve speak for himself.

No, study didn’t show that ‘mindfulness training doesn’t foster empathy or makes narcissists worse’

quick takes

Many of us have become accustomed to extravagant claims about the benefits of mindfulness that turn out to be based on poor quality studies with inadequate control groups. We become skeptical about what we are told about the benefits of mindfulness. We’ve come to expect a lot of confirmation bias.

Reactions to the study that I am going to be discussing, though, suggest that overexposure to these kinds of studies may create a bias of a different kind. Namely, we may have more accepting of claims of negative effects of mindfulness, even when they come a poor quality study.

A click bait headline of a link to an article in the British Psychological Society (BPS) Research Digest kept showing up in my Twitter feed.

bps on empthy study

mindfulness doesn't foster empahtyI might be inclined to believe it, without examining the evidence. Why not? It’s not at all clear that there are any specific effects of mindfulness, in the active ingredient, beyond nonspecific – placebo – conditions with which it is administered.

I suspect a lot of people who were retweeting it probably didn’t bother to check whether the article actually made sense.

When I will I obtained the open access study that had inspired the story in the BPS Research Digest, I could quickly see that the claims were not warranted.

Ridderinkhof A, de Bruin EI, Brummelman E, Bögels SM. Does mindfulness meditation increase empathy? An experiment. Self and Identity. 2017 Jan 3:1-9.

The abstract

Cultivating empathy is a presumed benefit of mindfulness, but this possibility has rarely been investigated experimentally. We examined whether a five-minute mindfulness exercise would cultivate empathy relative to two equally brief control exercises: relaxation and mind-wandering. We further examined whether mindfulness would be especially beneficial for people with autistic or narcissistic traits. Results showed no effect of mindfulness relative to both control conditions on mind reading, empathic responding, or prosocial behavior. Mindfulness effects were independent of autistic traits. Unexpectedly, people higher in autistic traits did show increased prosocial behavior across conditions. Intriguingly, mindfulness improved mind reading in non-narcissistic people, but reduced it in narcissistic people. These findings question whether a brief mindfulness exercise is sufficient for building empathy.

The study found no overall effects of mindfulness on empathy as it was measured in the study. The click bait headline was based on post hoc subgroup analyses.

When I drilled down into the article itself, I saw that it was not actually conventional mindfulness training that was provided to participants assigned to that condition, but a five-minute analog exercise.

Apparently the five-minute exercise was not very convincing to participants, because those who received it rated it as leaving them less mindful than those receiving a relaxation control manipulation rated themselves after it.

The authors nonetheless provided subgroup analyses organized around to personality variables, which they termed autistic traits and narcissism. Most of these analyses did not produce a significant effect, some were counterintuitive but the abstract and the authors discussion centered on the few of them

The authors fell into the trap of being swayed by the mere name of the measure. When is administered to a general population sample, the autism spectrum measure does not distinguish people who are more or less likely to exhibit autism spectrum characteristics. Similarly for the measure of narcissism. In each, the authors were interpreting small differences on the low end of the scale as if they were occurring at the high end.

So, the authors tackled reasonable question about whether mindfulness fosters empathy, but they did so with very weak methods. When they didn’t get positive results, they performed lots of subgroup analyses and cherry picked a few to overinterpret.

I follow some Twitter accounts because I expect them not only to alert me to findings to which I should pay attention, but because they also filter out things I should simply ignore. In this respect, the BPS Research Digest failed me. The click bait headline was simply misleading, but it did succeed in getting me and others to go to the website. The article acknowledge some of the problems of the study, but seem to dismiss them. Worse, the BPS article offered causal interpretations of what were undoubtedly cherry-picked, spurious effects.

We’re all suckers for believing effects that can be explained, even when the effects are not there.

What Jon Kabat-Zinn’s trademarking of MBSR means for mindfulness training.

  • BLOG_ArmingIntrospection-LEADFULL-Jon Kabat-Zinn has trademarked MBSR, legally restricting the use by others of the acronym for Mindfulness Based Stress-Reduction without his licensing.
  • Trademarking a treatment is unusual but can be worth millions in the multi-billion dollar business of mindfulness.
  • The mindfulness industry is increasingly dominated by vertical monopolies in which entrepreneurs dictate the need for mindfulness products and who can deliver them. Trademarking is a key tool in doing this.
  • Jon Kabat-Zinn’s trademarking of MBSR allows him to dictate its use in training drone pilots to be more effective in their airstrikes and to experience less guilt and PTSD. If you think this is an outrageous statement, read on.

Trademarking of acronyms for therapeutic interventions is highly unusual and usually undertaken to protect commercialization. Francine Shapiro attracted considerable negative attention when she trademarked EMDR.

What we can learn from the trademarking of EMDR

EMDR is a four letter acronym that stands for Eye Movement Desensitization Reprocessing. This four-letter combination is trademarked much like 3M or Coca Cola are trademarked. The actual therapeutic modalities involved in EMDR cannot be trademarked. (You can trademark a name but not a surgery!)

Wikipedia notes

EMDR has generated a great deal of controversy since its inception in 1989. Shapiro was criticized for repeatedly increasing the length and expense of training and certification, allegedly in response to the results of controlled trials that cast doubt on EMDR’s efficacy.[19][20] This included requiring the completion of an EMDR training program in order to be qualified to administer EMDR properly, after researchers using the initial written instructions found no difference between control no-eye-movement control groups and EMDR-as-written experimental groups. Further changes in training requirements and/or the definition of EMDR included requiring level II training when researchers with level I training still found no difference between eye-movement experimental groups and no-eye-movement controls and deeming “alternate forms of bilateral stimulation” (such as finger-tapping) as variants of EMDR by the time a study found no difference between EMDR and a finger-tapping control group.[19] Such changes in definition and training for EMDR have been described as “ad hoc moves [made] when confronted by embarrassing data” [35]

Dependence on unregulated certificates for credibility and marketing

Many persons, perhaps most, who seek training in psychotherapy or mindfulness do not have protected titles like clinical psychologist or psychiatrist. If they want to advertise as a counselor or therapist or mindfulness trainer,  they cannot point to any licensure. They can advertise certification or having studied with so and so, which is not regulated.

Marketers selling a training in a trademarked treatment have to be careful to avoid legal action:

In our certification classes we refer to this as REM so it is clear that we are not implying that we are teaching EMDR. We teach it a bit differently than the EMDR “sect” does! EMDR/REM is a protocol of hypnotic-like processes, which can be fairly useful in many cases.

The brilliant marketing strategy of Jon Kabat-Zinn

 Jon Kabat-Zinn has carved out an unusual, if not unique brand. He advertises his pedigree of having been trained by Buddhists, but denies that what is offering is in any way religious.

Kabat-Zinn was first introduced to meditation by Philip Kapleau, a Zen missionary who came to speak at MIT while Kabat-Zinn was a student. Kabat-Zinn went on to study meditation with other Buddhist teachers such as Thích Nhất Hạnh and Seung Sahn.[4] He also studied at the Insight Meditation Society and eventually also taught there.[4] In 1979 he founded the Stress Reduction Clinic at the University of Massachusetts Medical School, where he adapted the Buddhist teachings on mindfulness and developed the Stress Reduction and Relaxation Program. He subsequently renamed the structured eight-week course Mindfulness-Based Stress Reduction (MBSR). He removed the Buddhist framework and eventually downplayed any connection between mindfulness and Buddhism, instead putting MBSR in a scientific context.[4] He subsequently also founded the Center for Mindfulness in Medicine, Health Care, and Society at the University of Massachusetts Medical School. His secular technique, which combines meditation and Hatha yoga, has since spread worldwide.[4] The course aims to help patients cope with stress, pain, and illness by using what is called “moment-to-moment awareness.”[5][6]

Kabat-Zinn was born Jewish but has stated that his beliefs growing up were a fusion of science and art.[13] Although he has been “trained in Buddhism and espouses its principles,” he rejects the label of “Buddhist,”[7] preferring to “apply mindfulness within a scientific rather than a religious frame.”[4]

The branding of what Jon Kabat-Zinn offers is that it is science rooted in Buddhism. He can require weekend silent retreats for certification. There is no evidence that such retreats improve skills of mindfulness practitioners or the outcomes with patients that they achieve. Retreats are a spiritual, not an evidence-based practice.

But there is an intense competition among mindfulness trainers for bragging rights about how many weekends of silence they have endured on their journey to become what they are marketing.

The place of trademarking in the vertical monopoly of mindfulness entrepreneurship.

Mindfulness is a multi-billion dollar business and trademarking can be worth many millions.

Increasingly, mindfulness is dominated by vertical integration or vertical monopolies. Promoters offering product occupy key positions on governmental and quasi-governmental committees defining the need for mindfulness products and controlling who is authorized to deliver trainings and direct services. Witness Mindful Nation UK Mindfulness All-Party Parliamentary Group. The Oxford Mindfulness Centre had a key role in writing this report and offers products, including mindfulness retreats required to receive monies.

Capture

Trademarking MSBR can control the lucrative market of mindfulness training of drone pilots

Jon Kabat-Zinn is one of three Buddhists advising the US Department of Defense in the training of drone pilots to be more focused in their drone strikes and more inoculated against guilt and PTSD that might arise, particularly with the inevitable deaths of civilians.

jon Kabat Zinn lecturing

In a talk to the military:

The beauty of the mental training is that you can do it all day long,” said Kabat-Zinn. “You can bring awareness to everything you are doing. And then, the whole day in some sense becomes your mental training, becomes your mindfulness practice … It’s something virtually all human beings can do. It’s not luxury to practice mindfulness. It’s absolutely a necessity.”

Note that the following description of the DoD training program incorporates “Buddhist mindfulness based stress reduction courses.” Using an acronym introduces Jon Kabat Zinn’s trademark, ‘Buddhist MBSR courses.’

There are publicity photographs of marines sitting cross-legged practising meditation and equanimity with a rifle slung over their back, ready for use. Army psychologists believe that mindfulness and meditation will reduce Post Traumatic Stress Disorder (PTSD) for soldiers who kill or maim men, women and children or witness fellow soldiers being killed or maimed or suffer personal terror or anguish in the killing fields. The US military believe that their version of the Buddhist mindfulness based stress reduction courses can help make soldiers and drone pilots relax and stay immune from the emotional impact of their actions to kill and harm people.

Military intelligence officer, Elizabeth Stanley wrote a paper for the Defence Advanced Research Projects Agency (DARPA), stating that the meditation practises of Buddhist monks were necessary for drone pilots coping with information overload and for soldiers on the ground.

The MMFT website states that the course is designed to “improve resilience and mission effectiveness in other high-stress environments.”

Drone pilots in Nevada use their mindfulness training to stay focussed hour after hour, through the drones’ cameras, on Muslims visible outdoors, occupants in cars and on the streets. When they believe they have spotted their target(s) they press the Death Button besides the keyboard of Death TV.

Jon Kabat-Zinn can insist that it is vitally important for trainers to get prepared in his trademarked MBSR. Accept no substitutes.

 

 

The RCT of tango dancing versus mindfulness meditation for depression

giphy1If someone is unhappy with their life, should they take up mindfulness meditation or some other activity?

That is a question that a lot of people confront. For an answer, they might even look to this RCT comparing tango dancing lessons to mindfulness to a waiting-list control.

The authors actually specifically asked whether tango dancing is as effective in reducing stress, anxiety, and depression as mindfulness meditation. Furthermore, they raise the question whether tango dancing increased mindfulness more than meditation itself.

The authors did not actually produce a direct answer to the questions they raised, but what happened is interesting to me, even if my interpretation is counter to the authors.

In the end, if this RCT provides any answers at all, it is “if you’re trying to decide what to do to resolve unhappiness, you might want to select an activity that you enjoy participating in, for itself.” And I really don’t think you need an RCT to guide that decision.

But further investigating this RCT can provide some useful insights about interpreting such studies, because there are many out there like it.

Certainly one take away lesson is that we should pay more attention to the CONSORT flowchart when we began evaluating a report of an RCT than we typically do.

The citation and the access to a PDF of the study is directly below.

Pinniger R, Brown RF, Thorsteinsson EB, McKinley P. Argentine tango dance compared to mindfulness meditation and a waiting-list control: A randomised trial for treating depression. Complementary Therapies in Medicine. 2012 Dec 31;20(6):377-84.

Objectives: To determine whether tango dancing is as effective as mindfulness meditation in reducing symptoms of psychological stress, anxiety and depression, and in promoting well-being. Design: This study employed analysis of covariance (ANCOVA) and multiple regression analysis. Participants: Ninety-seven people with self-declared depression were randomised into tango dance or mindfulness meditation classes, or to control/waiting-list. Setting: classes were conducted in a venue suitable for both activities in the metropolitan area of Sydney, Australia. Interventions: Participants completed six-week programmes (1½h/week of tango or medita-tion). The outcome measures were assessed at pre-test and post-test. Main outcome measures: Depression, Anxiety and Stress Scale; The Self Esteem Scale; Satisfac-tion with Life Scale, and Mindful Attention Awareness Scale. Results: Sixty-six participants completed the program and were included in the statistical anal-ysis. Depression levels were significantly reduced in the tango (effect size d = 0.50, p = .010), and meditation groups (effect size d = 0.54, p = .025), relative to waiting-list controls. Stress levels were significantly reduced only in the tango group (effect size d = 0.45, p = .022). Attending tango classes was a significant predictor for the increased levels of mindfulness R2 = .10, adjusted R2 = .07, F (2,59) = 3.42, p = .039. Conclusion: Mindfulness-meditation and tango dance could be effective complementary adjuncts for the treatment of depression and/or inclusion in stress management programmes. Subsequent trials are called to explore the therapeutic mechanisms involved.

There are a number of deficiencies in this abstract as a representation of the study, but let’s put them aside for now.

The take away message that seems to be intended is that in some respects the tango group and meditation groups similarly reduce depression, and the tango group increased mindfulness. On Twitter, a lot of attention was given to the interpretation that only the tango group reduced stress.

Ratonale for the study.

If you are interested, you can read this passage in its context, but the authors offer an interesting rationale for their study

Toneatto and Nguyen17 suggest that any activity requiring awareness of current experience is likely to interrupt an individual’s thoughts about their past and fears about the future, so potentially lessening association between nega-tive thoughts and possible affective symptoms. Tango is one activity that could achieve this goal, since it is an absorb-ing activity that requires significant skills acquisition,8,16 and an awareness of current experience.17 It also facilitates an involvement in music, exercise, and touch, all of which have previously been reported to be effective in alleviating psychological distress.18—20

The important details of the recruitment

 The paper is actually better than most in describing explicitly what participants were told who were invited to enroll in the study.

Participants were recruited via advertisements asking for volunteers with self-reported stress, anxiety, and/or depression. They were invited to participate in a RCT evaluating tango dance relative to mindfulness meditation or waiting-listed control. They were advised that they could be allocated to one of those three groups.

Participating in an RCT or any research, for that matter, usually involves assuming some burden in terms of having to complete assessments and not drop out except for some good reason. But it also involves the gamble of randomization. Participants may be seeking an experience that they might not otherwise have available, and risking not being assigned to it, but nonetheless feeling obligated to stick with it.

We will soon be seen that most participants in the study were more interested in getting tango lessons than in becoming more mindful. Any hopes of interpretable results for a randomized controlled trial could been doomed from the start by a lack of equipoise- different participant valuation of tango lessons versus mindfulness at the outset, or it could have emerged once participants actually gotten involved in these activities. Regardless, different preferences and differences in the ability of the two interventions to retain participants were ultimately fatal for an interpretable findings, at least in terms of the outcomes of anxiety, depression, and stress.

The CONSORT flowchart

 consort flow tango

Most readers will skip over the CONSORT flowchart for these kind of studies. But it is good that most journals have accepted the requirement that reports of RCTs have such a flowchart. What is revealed about this study is that there were so many dropouts from participants assigned to mindfulness, much more that so than the tango group.

What is also interesting is that the waiting list group stuck around for assessments. I might’ve predicted ahead of time that assignment to this treatment would have the greatest number of dropouts because of the delay in getting any reward.

Inappropriate analyses.

The most appropriate and informative analysis for an RCT includes results for all participants who originally enrolled. That is an intention to treat analysis. Less preferable is what these authors did, an analysis limited to the participants available at follow-up.

In a situation of such different abilities of conditions to retain participants, many researchers would have resorted to some kind of imputation of missing data from what data were available. Such imputation typically assumes that loss to follow-up is random, which is unlikely. Aside from violation of this assumption, there is simply too few participants retained in the mindfulness group to provide a basis of guessing the outcomes of participants lost to the study. So, intent to treat analyses based on imputing outcomes to participants who were no longer available would be invalid.

But what the investigators did instead was similarly invalid. They ignored that a substantial portion of the participants assigned to mindfulness meditation were no longer available and analyzed only the results of those who stuck around. Consider this analogy of a climb up Mount Everest as a way of reducing stress. Most participants would make it, but would you make a decision about the value of attempting to climb Mount Everest based on results only from those who made it?

The voucher

 I did not pay attention to a key statement in the description of recruitment that later took on particular importance in interpreting what happened in the study. The statement was

What we were later told in the discussion section:

Finally, 97% of participants in the study chose to receive a tango dance voucher after the study, rather than a meditation voucher, suggesting the popular appeal of tango dance. This is an important consideration, since people tend to adhere to mindfulness-based programs more than other therapy approaches,22 especially if the experience is intrinsically positive,25 such as tango dance is reported to be.8

So, it appears that the participants recruited to the study had a strong preference for getting tango lessons, not mindfulness meditation. There was a broad dissemination of invitations to participate in the study. We’re not told, but maybe the researchers had to advertise so broadly, because they initially had difficulty obtaining participants. Maybe, –we are not the position to know–mindfulness was readily available in Sydney Australia, but tango lessons were in short supply. Participants enrolled in the study in hope of getting assigned to tango lessons.

Bad abstract

Having read the study, I could now see that the abstract was misleading. It left out the important detail of how many participants were recruited to the study instead focused on the number from whom results were available. Furthermore, I’m suspicious about the participants being described has self identified for depression. Most accurately, participants from were recruited for a study aimed at reducing anxiety, depression, and stress. They did not self identify as depressed, they simply filled out some self-report questionnaires. An even suspicious about depression being implicitly presented has the primary outcome. They could be confirmation bias here: it is the one of the three outcomes for which the results were strongest.

Back to the question of tango dance lessons versus mindfulness meditation

 Both dance lessons  and mindfulness meditation require commitment and practice to obtain benefit. I don’t think that participants enrolled in the study only to reduce their self-report measures. They expect to get something out of the experience. If nothing else, the dance lessons posed the prospect of leaving participants better able to dance. No guaranteed, of course, but it was in the offering.

So, back to a person pondering what to do with their life. The key issue is one of personal preference: do they think they are likely to engage in the activity enough to get any benefit? If the activity was supposed to produce something tangible, like dancing skills, would they be content with that alone? If a participant were only looking to have a bit of fun, would they really need the guidance of an RCT?

 

 

 

 

McMindful: Make money as a mindfulness trainer, no background or weekend retreat required.

McMindful: Make money as a mindfulness trainer, no background or weekend retreat required.

With an interview with

hustleCan a clinical psychologist ethically offer a product with improbable, unsubstantiated claims to be applied to patients by persons who have not been vetted for competence or fitness to treat patients?

A web-based training package promises to turn anyone quickly into a mindfulness trainer, regardless of background or previous training.

Promoters of the package claim it is backed by more science than its competitors.

There are no legal restraints in most jurisdictions on someone calling themselves a mindfulness trainer, coach, or therapist. No training requirements or background check.

There no enforceable ethical codes applicable to such persons once they hang out their shingles.

Many treatment settings are replacing therapists with mindfulness trainers.

Many persons with serious mental health problems seek mindfulness training, but this training does not prepare trainers to recognize and refer such persons.

I didn’t act quickly enough to a series of frantic emails from Seph Fontane Pennock Positive Psychology Program, and so I missed out on a deep discount for an exciting offer to become his next success story.

If I had been quicker. I could have received a 40% discount on a $750 downloadable training package that promised to turn anyone into a money-making mindfulness trainer, without them having to acquire any background or participate in a weekend retreat. It did not matter if a purchaser did not have any clinical background, because the program would release “the real trainer, teacher and coach in yourself that you’ll be proud of.”

My final invitation to become mindfulness trainer came in a breathless gushy, seemingly personalized email that began:  “Hey Jim, I’m blown away by all the emails about the success our members have started to see..”

It continued with testimonials from purchasers who were impressed that they could customize the materials to appear to be their own, including by putting their company logo on them.

The wannabe trainer doesn’t even need to study the package before slapping on a relabeling and selling to clients and industry.

The website makes it clear that it is superior to other training because it is better rooted in science. But just what does “rooted in science” mean? Is that as vague and meaningless as saying that performance of your automobile is rooted in physics? I think claims about the efficacy of interventions needed to be rooted in randomized trials or program evaluation and there is no evidence that this package has been put to these kind of tests.

The package is billed has instantly turning purchasers into mindfulness trainers.

You can simply take this, go out and teach mindfulness …

No longer will you have to go from A to B, from B to C, etc. Instead, you can go straight from A to Z. Mindfulness X is the ultimate shortcut.

It is claimed that professionals will be able to “instantly and successfully teach mindfulness.”

Who is the mastermind behind Mindfulness X?

Dr. Hugo Alberts (Ph.D.) describes himself as a “professor, entrepreneur and coach” who has touched the lives of thousands. With Mindfulness X, he had become a sought after trainer, but decided to stop live presentations in order to touch even more lives with this downloadable product.

I did some fact checking. Hugo (H.J.E.M.)Alberts, Ph.D. is an Assistant Professor in the Clinical Psychological Science Department at Maastricht University. Web of Sciences lists 19 publications for him, including a couple of low quality, underpowered studies of mindfulness.

Most importantly, I find no evidence of any peer-reviewed evaluation of Mindfulness X. The key issue is that Alberts is claiming extraordinary efficacy for this program. If his claims are true, it is more effective than any psychotherapy. Extraordinary claims require….

Elsewhere I have provided continually updated evaluations of mindfulness-based training and therapies. There is still a lack of evidence of any advantage of mindfulness over other active treatments. Claims about mechanism depend on low quality studies that do not rule out anything beyond nonspecific –placebo- effects. There may be no specific mechanism beyond that.

Mindfulness training is mostly a benign treatment, often delivered to persons who are lacking moderate to severe psychological problems. But it can have adverse effects on persons suffering simple or complex PTSD, ruminative chronic depression, or psychosis.

An increasing proportion of the treatment or coaching of persons with serious psychological problems is being done by persons lacking in any protected title or any independent certification of qualifications.

Such providers are not bound by enforceable ethics codes.

My advice to Dr Albert: you are quite junior. If you are serious about your scientific career, concentrate on producing quality research, not so much on making money in ways that threaten perceptions of your integrity. I assume you are a clinical psychologist. You have a responsibility to stick to evidence-based claims and to avoid the harm of turning loose on the community ill-trained or untrained promoters of mindfulness, particularly with vulnerable clients.

Dueling neurononsense: mindfulness training and gray matter

Confronting claims about mindfulness training changing the brain with claims that exposure to parks changes the brain suggests that neither claim had evidence.

Exhibit A

Mindfulness meditation training changes brain structure in eight weeks

mindfulness grey matter

participating in mindfulness.PNG

“Although the practice of meditation is associated with a sense of peacefulness and physical relaxation, practitioners have long claimed that meditation also provides cognitive and psychological benefits that persist throughout the day,” says Sara Lazar, PhD, of the MGH Psychiatric Neuroimaging Research Program, the study’s senior author. “This study demonstrates that changes in brain structure may underlie some of these reported improvements and that people are not just feeling better because they are spending time relaxing.”

What was the evidence for  these claims?

For the current study, MR images were taken of the brain structure of 16 study participants two weeks before and after they took part in the 8-week Mindfulness-Based Stress Reduction (MBSR) Program at the University of Massachusetts Center for Mindfulness. In addition to weekly meetings that included practice of mindfulness meditation — which focuses on nonjudgmental awareness of sensations, feelings and state of mind — participants received audio recordings for guided meditation practice and were asked to keep track of how much time they practiced each day. A set of MR brain images were also taken of a control group of non-meditators over a similar time interval.

How were these differences interpreted?

“It is fascinating to see the brain’s plasticity and that, by practicing meditation, we can play an active role in changing the brain and can increase our well-being and quality of life.” says Britta Hölzel, PhD, first author of the paper and a research fellow at MGH and Giessen University in Germany. “Other studies in different patient populations have shown that meditation can make significant improvements in a variety of symptoms, and we are now investigating the underlying mechanisms in the brain that facilitate this change.”

Amishi Jha, PhD, a University of Miami neuroscientist who investigates mindfulness-training’s effects on individuals in high-stress situations, says, “These results shed light on the mechanisms of action of mindfulness-based training. They demonstrate that the first-person experience of stress can not only be reduced with an 8-week mindfulness training program but that this experiential change corresponds with structural changes in the amygdala, a finding that opens doors to many possibilities for further research on MBSR’s potential to protect against stress-related disorders, such as post-traumatic stress disorder.” Jha was not one of the study investigators.

In general, it is not reasonable to make a big deal of a study comparing only 16 participants practicing mindfulness to 16 people who did not. Such a study is much too small to yield reliable, replicable differences. To be statistically significant in such a small study, differences between the two groups would have to be quite large. Mostly, large differences between groups found in small studies are exaggerated.

Even if moderate differences in grey matter were  associated with an intervention, the likelihood of them being shown in such a small study is considerably less than 50%. Meaning? That  these are most likelyspurious findings, even if real effects actually existed.

Even if such differences could be shown (and they were not reliably shown in this study, we wouldn’t know if anything specific about mindfulness caused them.

To make a fuss about these differences in terms of the function of these regions of the brain is silly.

Exhibit B: And now for the contrasting study.

Mental health: Parks and urban spaces can protect against mental decline in pensioners

parks protect

pic parks protect

Being surrounded by greenery was good for grey matter, as it was calming and quieter, according to the first study to look at the impact of parks on the mental health of pensioners.

Walking between busy urban environments and green spaces triggers changes in levels of excitement and engagement and reduces frustration.

The findings have important implications for architects, planners and health professionals with gardens and parks coming under pressure from austerity as we deal with an ageing population.

What was the evidence for these claims?

As part of the experiment, eight volunteers aged 65 and over from a wider sample of 95 wore an EEG (electroencephalogram) headset which recorded their brain activity as they walked.

The researchers also ran a video of the routes, asking the participants to describe ‘snapshots’ of how they felt. The volunteers were also interviewed before and after.

The study published in the International Journal of Environmental Research and Public Health found as well as preferring green space, it was beneficial to the volunteers’ brain health by being calming and quieter.

Co-author Dr Sara Tilley, of the University of Edinburgh, added: “To help ensure living longer is a positive experience for everyone, we need evidence based solutions to support lifelong health and well being.

Well, maybe, but this is not the kind of evidence that is needed. Numbers from such a small study are not evidence. But was the interpretation that was given?

“These findings – and others from the same project which show how important places are for our personal and cultural memories, and for enabling us to stay connected socially – have implications for the way we design for people of all ages, backgrounds and abilities so going outdoors in younger years becomes a lifelong passion for getting out and about.”

They mirror the findings of the 2013 study of younger adults from Heriot-Watt University, Edinburgh, which also found being surrounded by green reduced frustration and increased engagement.

Those researchers who also tracked brainwaves said parks affect the mind in a similar way to meditation.

I do not think either news item provides much evidence. We shouldn’t ignore the lack of sound evidence in comparing the two, but if we do, we have to say whatever is going on, is probably not very specific. But I doubt we have evidence that anything is going on that we did not have before these studies were introduced.

By the way, the parks-affect-the brain news item got more nonsensical if we had kept going:

A 2015 study of about 2,000 seven to 10 year old primary schoolchildren in Barcelona showed they improved memory and reduced inattentiveness in schools.

The findings may partly be explained by reduced exposure to traffic pollution, experts believe.

Other influences could include the psychological effect of having views of fields and trees rather than roads and buildings.

I will soon be offering e-books providing skeptical looks at mindfulness and positive psychologyeBook_Mindfulness_345x550, as well as scientific writing courses on the web, as I have been doing face-to-face for almost a decade.

Sign up at my new website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites.  Lots to see at CoyneoftheRealm.com.