VanderWeele TJ, Li S, Tsai AC, Kawachi I. Association Between Religious Service Attendance and Lower Suicide Rates Among US Women. JAMA Psychiatry. 2016;73(8):845-851. doi:10.1001/jamapsychiatry.2016.1243.
This recent study in JAMA Psychiatry received an extraordinary amount of attention in a short time, undoubtedly orchestrated by the journal. Prestigious journals interested in keeping their prestige precisely track how much immediate attention papers get using altmetrics.
The intent is to boost immediate attention, which increases early citations. In turn, early citations raise impact factors. Journal impact factors are calculated based on the number of citations the papers received two years after publication. Journals can advertise higher impact factors which boost subscriptions and paid advertisements. Here are the altmetrics for this article. They suggest it is an outlier in terms of the amount of attention it got in a short time.
Among the numerous sources of attention were religious oriented media. Catholic Online proclaimed How the Church can improve your life. But secular medical sources simply proclaimed Take Me To Church: Attending Religous Services Linked To Lower Suicide Rates Among Women.
An article in the UK Spectator, People who go to church live longer. Here’s why was written by the first author of the JAMA Psychiatry paper, but I had to compare authorship of the articles to discover this.
During such publicity campaigns, journals often temporarily provide free access to what would otherwise be pay walled articles in order to stimulate attention. Unfortunately, that wasn’t the case with this article, and so readers without access to a subscription or a university library could only check the claims against an abstract, not the full paper. Here’s an excerpt:
Design, Setting, and Participants We evaluated associations between religious service attendance and suicide from 1996 through June 2010 in a large, long-term prospective cohort, the Nurses’ Health Study, in an analysis that included 89 708 women. Religious service attendance was self-reported in 1992 and 1996. Data analysis was conducted from 1996 through 2010.
Results. Among 89 708 women aged 30 to 55 years who participated in the Nurses’ Health Study, attendance at religious services once per week or more was associated with an approximately 5-fold lower rate of suicide compared with never attending religious services (hazard ratio, 0.16; 95% CI, 0.06-0.46). Service attendance once or more per week vs less frequent attendance was associated with a hazard ratio of 0.05 (95% CI, 0.006-0.48) for Catholics but only 0.34 (95% CI, 0.10-1.10) for Protestants (P = .05 for heterogeneity). Results were robust in sensitivity analysis and to exclusions of persons who were previously depressed or had a history of cancer or cardiovascular disease. There was evidence that social integration, depressive symptoms, and alcohol consumption partially mediated the association among those occasionally attending services, but not for those attending frequently.
An accompanying editorial
Koenig HG. Association of religious involvement and suicide. JAMA Psychiatry. 2016 Jun 29.
had free access. It extolled the virtues of the study:
The study by VanderWeele et al is important because of the large sample, lengthy follow-up period, and rigorous statistical methods used to analyze the data, including adjustments for baseline religious service attendance and removal of women who were previously depressed or with major physical health problems. Adjustment for baseline religious attendance and initial removal of women with depression or physical illness is particularly important to avoid the problem of reverse causation, an issue in studies of religious service attendance and other health outcomes. Depressed persons at greatest risk for suicide are often socially withdrawn and less likely to attend religious services, which could otherwise explain the association.
The editorial suggested immediate clinical applications of the findings, but ended on a discreet note of caution that would set off alarm bells for a skeptic.
What should mental health professionals do with this information? Evaluating patients’ moral beliefs about suicide and level of involvement in religious community may help clinicians gauge risk of suicide. Thus, the findings by VanderWeele et al underscore the importance of obtaining a spiritual history as part of the overall psychiatric evaluation, which may identify patients who at one time were active in a faith community but have stopped for various reasons. Exploring what those reasons were, particularly among the socially isolated, and perhaps supporting a return to such activity, if the patient desires, may help produce social connections that lower suicide risk. If based exclusively on the findings of VanderWeele et al, these suggestions would apply only to white female nurses in the United States. Given the large amount of research in other ethnic, sex, socioeconomic, professional, and nonprofessional groups that shows similar associations, one might be tempted to apply these findings to other populations as well. Nevertheless, until others have replicated the findings reported here in studies with higher event rates (ie, greater than 36 suicides), it would be wise to proceed cautiously and sensitively.
This last sentence provides crucial information that should have been reported in the abstract of the article. The seemingly impressive study involved predicting only 36 suicides. Any multivariate analyses spread these 36 suicides across little boxes of categorical variables. Imprecision in the measurement of any of these variables or any misclassification could produce very different results.
As a skeptic accustomed to hard-sell efforts based on weak data, this is the information I would have immediately sought. Armed with it, I would’ve been prepared to reject as nonsensical the pseudo-precision of a dramatic estimates of effacts that are contained in the abstract..
…Attendance at religious services once per week or more was associated with an approximately 5-fold lower rate of suicide compared with never attending religious services (hazard ratio, 0.16; 95% CI, 0.06-0.46).
Come on! This “five-fold” difference refers to differences in the distribution of fewer than 36 suicides in a couple of boxes. It would have been less striking if it were reported in absolute terms. The boxes are “no attendance,” versus “attendance once a week or more.” That means that some of the already small number of 36 suicides were thrown out because they did not occur among women who attended church but less consistently than once a week or more.
Dropping subjects in the middle of a distribution and focusing on extremes inflates the significance of results. There is no evidence that these authors made the decision to do so without first peeking at their data.
Seemingly impressive multivariate analyses that build on these analyses just add further nonsense and noise.
Women who have the regularity of routine to show up in church at least once a week or more are often regular in other ways they can affect the health and well-being.
The first is called “healthy user bias.” As Gary Taubes described nicely, “people who faithfully engage in activities that are good for them — taking a drug as prescribed, for instance, or eating what they believe is a healthy diet — are fundamentally different from those who don’t.
Women who regularly go to church take better care of themselves in still other ways.
Next, there is another subtle component of healthy-user (or “healthy continuer”) bias. This is the “compliance or adherer effect or bias”. Individuals who comply or adhere with their doctors’ orders when given a prescription are different and healthier than people who don’t.
This Nurses Health cohort is well studied. In lots of papers. Its many biases have been pointed out. To start with, it’s limited to nurses. Extremes of social deprivation are excluded. And the rates of suicide in the Nurses Health Study are substantially less than in the general population of women
I don’t have much confidence in the claims being made in the press coverage or the article itself.But my skepticism turned to disappointment in the authors. I noticed that they did that did not adequately acknowledge results concerning suicide from the same Nurses Health Study set that were published less than a year ago in the same journal.
Tsai AC, Lucas M, Kawachi I. Association between social integration and suicide among women in the United States. JAMA Psychiatry. 2015 Oct 1;72(10):987-93.
Unlike the present study, this one is available as a PDF without a pay wall. For reasons that are explained, it has 43 deaths by suicide to explain and suggests that social integration is a reasonably good predictor of death by suicide. This is a contradiction of what is said in the current paper.
Who knows why. But dropping a few death by suicide can make a substantial difference when there are so few to begin with. Then there is always the voodoo of applying multivariate statistics in predicting so few infrequent events. I think this further demonstrates the problems of making a big fuss when talking about so few suicides. Add or subtract only a few suicides and you get a whole different story to tell.
I hope you have learned some things from this exercise:
- Whenever you see reports of epidemiological studies of suicide, keep in mind the infrequency of suicide. Pay attention to the number of death by suicide to explain, not the size of the overall sample.
- Correlation does not equal causality, particularly when the correlations occur with multivariate statistical analyses with unknown peeking at the data ahead of time.
- Beware of clinical implications being drawn from such weak data.