Abstract of a talk to be given at the Australian National University (ANU), room G08, Building 39, 3pm September 11, 2014.
UPDATE (October 24, 2014): Here is a link to the Powerpoint presentation and to a YouTube interview with ANU Research School of Psychology’s Professor Mike Smithson. A list of relevant links to James Coyne’s blogs has also been added at the end of this post.
In 2005, John Ioannidis made the controversial assertion in a now famous PLoS Medicine paper that “Most Published Research Findings are False”. The paper demonstrated that many positive findings in biomedicine subsequently proved to be false, and that most discoveries are either not replicated or can be shown to be exaggerated. The relevance of these demonstrations was not appreciated until later in psychology.
Recent documented examples of outright fraud in the psychological literature have spurred skepticism. However, while outright fraud may be rare, confirmatory bias and flexible rules of design and analysis are rampant and even implicitly encouraged by journals seeking newsworthy articles. Efforts at reform have met with considerable resistance, as seen in the blowback against the replicability movement.
This talk will describe the work of one loosely affiliated group to advance reform by focusing attention not only on the quality of the existing literature, but on the social and political processes at the level of editing and reviewing. It will give specific examples of recent and ongoing efforts to dilute the absolute authority of editors and prepublication reviewers, and instead enforce transparency and greater reliance on post-publication peer review of claims and data.
Optional suggested readings (I suggest only one or two as background)
UPDATED Blog posts coordinated with the slide presentation relevant to Are most positive findings in psychology false or exaggerated? An activist’s perspective.
Exposing Meta Analyses with an agenda: American anti-abortionist gets flawed meta Analysis into British Journal of Psychiatry
Negative Mental Health Effects of Abortion: Does getting an abortion damage women’s mental health? October 24, 2011
More on Review Claiming Abortion Hurts Women’s Mental Health: Author had conflict of interest, used scare tactics. November 15, 2011
Editor Should Have Caught Bias and Flaws in Review of Mental Health Effects of Abortion: Was publication of a flawed anti-abortion review deliberate? December 5, 2011
Is Having an Abortion Likely to Damage a Woman’s Mental Health?: Evaluating the Evidence from a Controversial Review. November 6, 2011
Going after bad science that misleads cancer patients
A formal request for retraction of a Cancer article May 12, 2014
Doubts a Classic Lancet Study Showed Psychotherapy Improves Survival of Cancer Patients. eptember 2, 2013
Frightening Breast Cancer Patients with Bad Science September 28, 2012
Pseudoscience of positive psychology and health
Challenging unfair editorial practices
Whomp! Using invited editorial commentary to neutralize negative findings. November 13, 2013.
I am holding my revised manuscript hostage until the editor forwards my complaint to a rogue reviewer. October 30, 2012
Exposures of conflicts of interests (COIs) associated with promoters of treatments
Soothing psychotherapists’ brains with NeuroBalm. June 24, 2014.
Are meta-analyses done by promoters of psychological treatments as tainted as those done by Pharma? May 20, 2014
Moving From Criticism and Activism to Proposals for Constructive Change
A. Democratizing Post-Publication Peer Review
Join PubMed’s Revolution in Post Publication Peer Review.October 22, 2013
B. Salvaging psychotherapy research
Salvaging psychotherapy research: a manifesto. June 10, 2014
What We Need to Do to Redeem Psychotherapy Research. June 10, 2014.