Why University College, London issued embargoed press release for an already published mindfulness study is more interesting than the insipid study itself.
The study is actually provides an excellent teaching example of what can happen when a recipe for hype-worthy mindfulness study is mindlessly followed.
University College, London should put more money into providing competent methodological and statistical consultations than orchestrating publicity campaigns promoting studies of mindfulness laced with pseudoscience.
A journalist sent me an embargoed press release from University College, London. At first I couldn’t make sense of why there was embargo, because the study itself was already available on the web.
Apparently UCL was trying to coordinate a media campaign for a mediocre study by coordinating churnaling – essentially plagiarizing by journalists – in the media like a chorus of quacking ducks..
Not surprising, sources, like The London Times and Time complied fabulously.
New sources churnaled the press release, which said:
Brief training in mindfulness strategies could help heavy drinkers start to cut back on alcohol consumption, finds a new UCL study.
After an 11-minute training session and encouragement to continue practising mindfulness — which involves focusing on what’s happening in the present moment — heavy drinkers drank less over the next week than people who were taught relaxation techniques, according to the study published in the International Journal of Neuropsychopharmacology.
“We found that a very brief, simple exercise in mindfulness can help drinkers cut back, and the benefits can be seen quite quickly,” said the study’s lead author, Dr Sunjeev Kamboj (UCL Clinical Psychopharmacology Unit).
“We used a highly controlled experimental design, to ensure that any benefits of mindfulness training were not likely explained by people believing it was a better treatment,” said co-author Dr Tom Freeman (Senior fellow of the Society for the Study of Addiction), who was part of the research team while based at UCL.
The mindfulness group drank 9.3 fewer units of alcohol (roughly equivalent to three pints of beer) in the following week compared to the week preceding the study, while there was no significant reduction in alcohol consumption among those who had learned relaxation techniques.
“Practising mindfulness can make a person more aware of their tendency to respond reflexively to urges. By being more aware of their cravings, we think the study participants were able to bring intention back into the equation, instead of automatically reaching for the drink when they feel a craving,” Dr Kamboj said.
The study itself was not very thoughtfully constructed according to a familiar recipe:
1. Appeal to neoliberal policies by setting out to claim that citizenry can learn to self-discipline themselves away from unruly behavior with cheap methods that will distract from underfunding of health and substance abuse services.
2. Take some speculations about how mindfulness applies eastern meditation techniques to change health behaviors and apply them to drinking alcohol.
3. Ignore existing conventional literature concerning skills relevant to avoiding excessive alcohol consumption.
4. Rely on effect sizes from an underpowered study of mindfulness for reducing smoking. Recruit exactly the same number of participants.
5. Assemble a battery of psychological measures that might conceivably be affected by mindfulness meditation or an imbalance of nonspecific effects between groups.
6. Assemble a set of biological measures claimed in the literature to be biomarkers for a state of contemplative mindfulness.
7. In response to criticism that past mindfulness studies have been unblinded comparisons of mindfulness to inert conditions, claim blinding of experimenters and participants and use an active treatment, relaxation.
8. Avoid measurement of whether participants actually practiced mindfulness.
9. Ignore indications from full study analyses whether within-group analyses would make for attractive results.
10. More generally, significance-chase and selectively report favorable results.
What can go wrong?
1. The authors could take advantage of lower standards of evidence required to promote results appealing to neoliberal prejudices. Yet they may have gotten carried away with claims only 11 minutes of mindfulness reduce drinking equivalent to a three pints of beer or a bottle of wine per week. And the adverse social consequences of these claims being accepted:
Weak designs with short-term subjective self-report outcomes proclaiming dramatic results can lead to continued underfunding of services and denial of more intensive services to those most in need.
2. Armchair speculations from the mindfulness literature are poor substitute for functional analysis of behavioral processes leading towards moderate versus excessive drinking. Really, let’s let some common sense in:
3. By the time that people are contemplating beer in their mouths, it may be too late for them to engage more adaptive and effective practices. Furthermore, concentrating too much on the taste of beer may be counterproductive for those who enjoy it and who would be better off focusing on distracting social cues. Namely, they could be concentrating on the conversation and the people around them drinking less, rather than getting focused on the beer in their mouth.
4. As expected, effect sizes from underpowered study in a different population and different clinical focus proved exaggerated. Moreover, given the study’s distinctive claim about elucidating the biological underpinnings of mindfulness, a much larger sample would’ve been needed to be geared to finding of facts in those outcomes if they were indeed present.
5. Many of the other psychological measures were poorly chosen and irrelevant to evaluating the intervention, but were nonetheless selectively presented. I won’t waste my readers time keeping score as I cut through this underbrush.
6. The assessment of particular biological correlates of mindfulness was a both a matter of convenience of measurement in a short session and belief in the myth that such correlates are not only biomarkers, but the mind-body indicators that participants had achieved a state of contemplative mindfulness in the process of listening only to 11 minute tape.
A biological correlates does not a biomarker make, and less specificity can be established and cross validated cut points for mindfulness verses-not mindfulness. What is also ignored is that there are no specific “biological markers” for mindfulness. Any correlates with mindfulness training are often due to nonspecific aspects of the treatment, if they are not noise.
Cardiac vagal tone is particularly dubious as a measure. It’s highly responsive to contextual factors and has large intra-and inter-individual variability. Moreover, it is susceptible to changes in breathing and so it is not surprising that changes were greater in the relaxation group. In general, it would take a much larger study under much more carefully controlled conditions to produce meaningful results.
The authors conveniently ignored that other than when conducted by promoters of transcendental meditation, reliable differences in blood pressure are typically not found. With such a grossly underpowered sample, any apparent differences are likely to be artifactual, due to outliers, or simply noise being interpreted as music..
7. The authors claim that a strength of their study is that they have a double-blinded comparison of two active treatments. Neither research assistants nor participants were told which was the mindfulness nor relaxation condition. Yet, the mindfulness tape gave much clearer instructions as to the experimenters intentions and expectations about how effects would be reported in subjective self-report measures. It’s not clear why participants would expect getting relaxed would reduce alcohol consumption.
The authors seem unaware of the literature concerning how easy it is to induce changes in subjective self-report of short-term alcohol consumption. In the 80s, a number of us did studies in which the active intervention was simply eliciting reports of alcohol consumption and giving participants feedback that it was more than peers rank or compared to some normative standard. I seem to recall that if you follow that literature back further, you’ll see that this active treatment had started as a control condition with significant effect on self-reported alcohol consumption in the short term. I’m very skeptical about whether such changes persist over the long-term, except in participants otherwise motivated to reduce the alcohol consumption.
8. Despite the assembly of a full range of psychological and biological measures, the authors fail to access whether participants actually went home and practice mindfulness. They don’t report any attrition, but that might be in effect a participants been able to report in remotely, rather than not showing up for reassessment as part of an otherwise noncompliance with the study.
9. The abstract of the published study clearly indicated the point at which the authors commit themselves to significance chasing and p-hacking.
Both groups showed acute reductions in craving after training, although a trend Group x Time interaction (p=0.056) suggested that this reduction was greater in the relaxation group (d = 0.722 P < .001) compared with the mindfulness group (d = 0.317, P = .004).
10. I could go on, but I think you get the picture. An overwhelming number of statistical tests within and between groups on a full range of often irrelevant variables are mustered to make the case that more of this sort of research is needed. The spin on the findings in the press campaign can only serve to encourage more of this research and strengthen the tendency towards confirmatory bias. But in the end, the authors hypothesized a group by time interaction in changes in cravings and did not obtain it, although admittedly in an underpowered study. But they made it into Time magazine and The London Times described in exactly the words they provided.
As an experiment, I did not provide my usual links to relevant literature. I’ve been disappointed in how few readers click through and actually access those links, despite my efforts to make many of them open access. However if any readers want to take issue with me, I invite them to provide links to studies at odds with what I’m saying and I will see what I can come up with in response.