Our BMJ paper wins the 2013 Bill Silverman Prize from the Cochrane Collaboration

cochraneThe Cochrane Collaboration has awarded my colleagues and me the honor of the 2013 Bill Silverman Prize. We are thus approved as “troublemakers.”

I find this recognition quite validating and encouraging of our efforts to improve the scientific literature and dissemination of the evidence it produces to clinicians, patients, and policymakers by promoting evidence-based skepticism.

The BMJ paper that prompted our consideration for the award is just one of many in which we take critical looks at the scientific literature, exposing crucial biases, exaggerated or simply false claims, and the inadequacies in the peer review process that produces them. Our goal is to disseminate a set of tools so that others can scrutinize not only this literature, but even our appraisal of it and come to their own conclusions.

According to the website for the Bill Silverman Prize, it

acknowledges explicitly the value of criticism of The Cochrane Collaboration, with a view to helping to improve its work, and thus achieve its aim of helping people make well-informed decisions about health carebill cochrane by providing the best possible evidence on the effects of healthcare interventions

The Bill Silverman prize is annually awarded to the authors of a paper that

  • was of high quality.
  • was accompanied by constructive suggestions on how the relevant aspects of the work of The Cochrane
  • Collaboration could be improved; and
  • has had, or is likely to have, a positive impact on the scientific quality,, relevance and use of Cochrane
  • Reviews.

Who was Bill Silverman?

According to the website for the prize

William (Bill) Silverman (1924-2004) was one of the founders of American neonatal medicine. He was honoured repeatedly as one of the pioneers in his specialty; however, he often evoked somewhat contradictory responses amongst his col leagues because he was in the habit of raising troubling questions about the scientific basis and ethics of his and their practices. Like many of the people who have helped establish the Cochrane collaboration, Bill Silverman could be regarded as a ‘troublemaker ‘. As he reiterated frequently, however, criticism is a form of troublemaking that can help drive progress. Furthermore, criticism should not be limited to examining the work of others, but should also include self-criticism.

What was our BMJ paper about?

You can read our open access BMJ paper here. As its title indicates, we were concerned about whether Cochrane reviews of drug trials took adequate consideration the sources of funding for these trials. It is well-established that trials funded by pharmaceutical companies consistently obtain better results for their own products than trials funded by other sources. We systematically reviewed the reviews of drug trials produced by the Cochrane collaboration in 2010.

Our conclusion:

Most Cochrane reviews of drug trials published in 2010 did not provide information on trial funding sources or trial author-industry financial ties or employment. When this information was reported, location of reporting was inconsistent across reviews.

One of the strengths of reviews conducted by the Cochrane Collaboration is they include formal assessments of the risk of bias of the clinical trials that are being considered, We argued that the criteria for risk of bias should be revised to include source of funding.

Don’t be surprised if you see a change in risk of bias criteria the near future.

I am greatly honored by our groups’ receipt of this prize, but I think it’s existence speaks to the exceptional openness of the Cochrane Collaboration to criticism and their nondefensive receipt of it.

You can keep up to date on our evidence-based skepticism by following this blog, as well as my postings at PLOS Mind the Brain and Science Based Medicine. You can also follow me on Twitter @CoyneoftheRealm