Recent reviews in British Journal of Health Psychology and Psycho-Oncology conclude that cancer patients live longer as a result of getting psychotherapy, a claim the American Cancer Society already considers discredited. The reviews arrived at this positive assessment by distorting the available evidence and outright ignoring some well-designed, but negative studies.
Claims of mind beating cancer have been around for a while. The entire Volume 2 Issue 3 of Psychological Inquiry was devoted to others’ investigations of claims by Ronald Grossarth-Maticek et al (1982) about the extraordinary effects on survival of what was termed ‘creative novation behavior therapy.’ The general conclusions were that the data showed one therapist providing an unbelievable amount of therapy, there were obvious errors in the recording of the data or maybe, as some suggested, the data were simply fraudulent. You can read more here.
David Spiegel was among the experts weighing in and he declared that the very size of the effect made the claims implausible, an opinion shared by others.
Yet, the thorough investigation and discrediting of Grossarth-Maticek and the publicity that it generated have still left the reputation of creative novation behavior therapy intact enough for it to be declared the most effective psychotherapy for extending the lives of cancer patients, at least in a review recently circulating in alternative medicine circles.
The idea that cancer patients can extend their lives through psychological methods has a powerful cultural appeal and the backing of strong vested interests that provide effective resistance against contrary evidence, even in more conventional scientific circles.
Seemingly credible claims were renewed by a report in Lancet of a study by Spiegel and colleagues of supportive expressive group psychotherapy. Ironically, the authors described the effect for their intervention as “consistent with, but greater in magnitude than those of Grossarth-Maticek et al. (1984).”
- Any differences between women receiving therapy and those in the control group were due to the inexplicable cluster of deaths among the few remaining control group that occurred 20 months into follow up. This odd drop made the control group significantly different from what would be expected from the larger population of women with metastatic breast cancer from which the group had been drawn. No one has ever been able to replicate this odd survival curve.
- Dramatic claims in the Lancet article that women receiving the psychotherapy lived twice as long depended on the wrong statistics, mean survival time. The more appropriate median survival time in both groups was about 20 months.
The Lancet study has nonetheless become the standard by which subsequent studies are judged as being successful replications versus unsuccessful failures. Replication of what is in the literature is good, not getting a positive finding is considered bad. Investigators attempting to publish transparently negative results run into trouble because of a strong publication bias enforced by vested interests.
The British Journal of Health Psychology and Psycho-Oncology articles deemed as successful replications of the Lancet study 7 other studies using widely different patient populations, methodologies, and interventions. Yet, each of them actually failed to demonstrate a survival effect for psychotherapy. In another blog post, I showed
- Two studies involved better nursing care, and were not intended to show whether psychotherapy alone could extend survival.
- Other studies involved providing different medical monitoring and care to patients assigned to psychotherapy group, so effects of psychotherapy alone could not be distinguished.
- One study only became positive with a favoring of counterintuitive results from unplanned subgroup analyses. The planned primary analyses were clearly negative.
- Another study involved different strategies for dropping patients from the intervention versus control group and voodoo statistics to claim a positive effect.
- One study claimed a survival effect, but presented only inappropriate multivariate analyses. When our peer-reviewed critique expressed skepticism whether simple analyses showed any effect, the authors refused an offer from the journal to reply or to discuss their results with journalists.
- A final study did not involve psychotherapy, but structured discussions of options and palliative care at the end-of-life that would become the basis of decisions about what medical care to receive. A better end of life, not a longer life was the goal. Patients in the control group got more aggressive care, but did not live as long as patients assigned to the intervention group.
There are still other relevant studies available in the literature that attempted to replicate the classic Lancet study, but that obtained negative results. The studies were suppressed in the recent reviews, but will be revealed in this blog. Finding them can be difficult, particularly if you rely on going first to the reference lists of reviews.
First, however, we need to understand the problems of attempting to find published replications in order to evaluate whether an established published finding is valid.
John Ioannidis explains why negative studies are hard to find
Stanford epidemiologist John Ioannidis has provided an excellent explanation of how trying to acknowledge failures to replicate go awry. It is difficult to publish transparently reported negative findings. As seen in the British Journal of Health Psychology and Psycho-Oncology reviews, such studies also often fail to get cited even when they do get published.
Ioannidis starts by noting the importance of replication
Replication is considered an essential feature of the scientific process for increasing the credibility of initially identified promising discoveries and improving the accuracy and precision of estimated effects. However the practice of replication is fraught with difficulties. In several scientific fields, funders, journals, and peers may create disincentives towards replication efforts, considering them second-rate, me-too efforts unworthy of funding and prestigious publication.
Transparent reporting of failed replications is thwarted by at least two means:
Obedient replication, where investigators feel that the prevailing school of thought is so dominant that finding consistent results is perceived as a sign of being a good scientist and there is no room for dissenting results and objections.
Thus, investigators can be shamed and intimidated by the power of the prevailing view and distort their findings to conform. Or they can be more directly intimidated by editors and reviewers:
Obliged replication, where the proponents of the original theory are so strong in shaping the literature and controlling the publication venues that they can largely select and mold the results, wording, and interpretation of studies eventually published.
Ioannidis introduces an element of realism into what have been idealized accounts of the corrective powers of replication
For a paper to get published, it may have to pay its dues to the prevailing theory. In fields where financial conflicts are also important, forced replication may take the form of production of supporting papers with ghost and gift authorship, e.g. when drugs or devices are involved. This could include both original data and supporting editorials and reviews. Financial conflicts related to industry products may not be as common in the psychological sciences as in medicine, but academic conflicts related to allegiance and confirmation biases and may be as strong and influential in shaping the literature eventually.
Return of the suppressed: Negative studies not cited in the British Journal of Health Psychology article [Updated* See end of blog post]
As you read through the following summaries, note the small number of times of each paper has been cited. Being cited, of course, means that people searching for papers can be led to a scientific paper from another. Citations are also a measure of prestige.
Kissane, D., & Li, Y. L. (2008). Effects of supportive-expressive group therapy on survival of patients with metastatic breast cancer: A randomized prospective trial. Cancer, 112, 443-444. Cited 7 times.
- This is a careful attempt at replicating Spiegel et al’s post hoc, counterintuitive subgroup analysis findings that women with estrogen receptor (ER)-negative women live longer with psychotherapy, even if there is no effect for other women. Kissane and Li had the advantage of considerably larger sample but could not find an effect.
- In clinical trials with otherwise negative findings, results from unplanned subgroups are generally not accepted until replicated. This makes Kissane and Li’s failure particularly important for the credibility of the previous results.
Boesen, E. H., Boesen, S. H., Frederiksen, K., Ross, L., Dahlstrøm, K., Schmidt, G., … & Johansen, C. (2007). Survival after a psychoeducational intervention for patients with cutaneous malignant melanoma: a replication study. Journal of Clinical Oncology, 25(36), 5698-5703. Cited 20 times
- A Danish study that attempted replication of Fawzy et al (1993) with 258 Danish patients with malignant melanoma.
- No evidence was found that psychoeducation did not increase survival or the recurrence-free interval.
Ross, L., Frederiksen, K., Boesen, S. H., Karlsen, R. V., Rasmussen, M. S., Sørensen, L. T., … & Johansen, C. (2009). No effect on survival of home psychosocial intervention in a randomized study of Danish colorectal cancer patients. Psycho‐Oncology, 18(8), 875-885. Cited 2 times
- Randomly assigned 249 Danish colorectal cancer patients undergoing abdominal surgery to a control or an intervention involving home visits with overall survival and selected immune parameters ass outcomes.
- The intervention group received 10 home visits from a project nurse or a medical doctor. The home visits aimed at providing emotional support and information.
- The study failed to find any evidence that the psychosocial intervention significantly affected survival or selected immune parameters.
The Danes have thus conducted two clinical trials to evaluate claims that psychosocial intervention extends survival. These studies represent some of the largest, best designed studies available. But because the hypothesis that they were attempting to replicate never had strong support anyway, they obtained negative results. Unlike most others, the Danes put no lipstick on what others my consider their pig, but persisted in efforts to get a transparent reporting of their findings published. For this effort, they were relegated to obscurity, eventually getting published, but virtually going largely uncited. The Danes just don’t play the game right.
Choi, J., Kuo, C. W. J., Sikorskii, A., You, M., Ren, D., Sherwood, P. R., … & Given, B. A. (2012). Cognitive behavioral symptom management intervention in patients with cancer: survival analysis. Supportive Care in Cancer, 20(6), 1243-1250
- Retrospectively explored the effect of a cognitive behavioral symptom management intervention on survival in individuals with solid tumors.
- Data were obtained from a randomized controlled trial that were originally designed to evaluate whether cognitive behavioral therapy to reduce symptom severity, not survival in 237 individuals with solid tumors during their first course of chemotherapy.
- Participants were randomized into: (1) ten-contact, 20-week CBT plus usual care (n = 118) or (2) usual care only (n = 119).
- Receipt of cognitive behavioral therapy was not associated with better survival.
This newer study is similar to other studies that were not originally designed to test whether psychological interventions affected survival, but that had the capacity for long-term follow up. The authors are clear that survival was not the primary outcome. Other studies claim positive effects on survival under similar circumstances and are likely to get published because they are positive studies, despite any statistical and methodological problems that preclude accepting their results.
We don’t know how many other studies are out there in which investigators after the fact evaluated whether they had produced a survival effect. When they don’t find one, they can simply abandon their effort has having pursued an improbable hypothesis anyway, or journal editors and reviewers can reject publication because they have a negative effect for outcome that had not been previously designated.
Overall, these negative studies are likely to remain obscurity, while inaccurately reported findings from small, methodologically inferior studies get attention because investigators knowingly or or unknowingly distort the results to appear positive. There is a strong double standard that favors studies claiming to be positive, even when they are not.
Failures to replicate existing findings can move the field away from bad hypotheses and allow the transfer of resources to more productive lines of inquiry. In the case of the hypothesis that psychotherapy extends survival, many who were originally enthusiastic about it have become skeptics. They have declared that it is time to give up for it and we should move on.
The problem of suppression of negative findings is widely recognized, and lots of reforms are being proposed. But as long as journals like British Journal of Health Psychology or Psycho-Oncology about biased reviews to enter the literature with no or inadequate peer review, the problem is going to persist. Cancer patients and their family caregivers and healthcare providers are going to be making decisions based on bad advice if they pay any attention to what is found in these kinds of journals.
*UPDATE: I have now found yet another large, Danish study that does not find an effect of psychological intervention on survival:
Boesen, E. H., Karlsen, R., Christensen, J., Paaschburg, B., Nielsen, D., Bloch, I. S., … & Johansen, C. (2011). Psychosocial group intervention for patients with primary breast cancer: a randomised trial. European Journal of Cancer, 47(9), 1363-1372.
- Randomly assigned 210 women with primary breast cancer to intervention or control group.
- intervention involved two weekly 6-h sessions of psycho-education plus eight weekly 2-h sessions of group psychotherapy.
- No effects on any of the psychosocial outcomes.
- Not enough cases of death to analyse overall survival, but at end of follow up, but six patients in the intervention group, three in the control group had died.