Blogging on bias
Publication bias became a big blogging topic last week as a PLoS Medicine paper was picked up by several influential sites. Lisa Bero and colleagues found that a quarter of trials submitted to the Food and Drug Administration between 2001 and 2002 in support of new drugs applications remain unpublished a year after the fact. The study also found that among the published results, unexplained discrepancies between the FDA submission and the published studies tended to lead to more favourable presentations of the drugs.
For the past two weeks it has been the most viewed paper on the PLoS Medicine website and discussion of the paper has passed from blog to blog, including:
– Wired (“Drug Companies Cook Books, Misleading Doctors”).
The Wired science blog, which has resulted in fifteen comments so far, highlights the discrepancy found in the study between results submitted to the FDA and those published in medical journals, and the key pieces of trial data that vanished in the published results. “The main thing that jumped out at me was the addition and deletion of primary outcomes. Those are the most important outcomes of a trial. To find that one disappeared from a paper, or just appeared in a paper, is pretty amazing to me,” Lisa Bero is quoted in the piece.
– Economist (“Absence of Evidence”).
There are twelve comments in response to the Economist’s question: “Do drug firms suppress unfavourable information about new products?” The print and online versions of the Economist also make reference to the perspective by An-Wen Chan that we published alongside the study, which argues that all key trials documents should be made public. “It has taken decades for trial registration and results disclosure to be implemented; hopefully, for the sake of patients, public access to full protocols and regulatory agency submissions will come much sooner”, says Chan in the perspective.
How else to fix the discrepancy between the trials submitted to the FDA and the published results? One of the more cynical comments about the research in response to the Wired blog ( “Hmmmm, big corps, unethical, self serving… TELL ME SOMETHING I DON’T ALREADY KNOW! Better yet, give me some suggestions regarding what I can do about it” ) is answered in Smooth Pebbles, a blog by David Dobbs, a freelance science writer for the New York Times amongst others. He quotes Lisa Bero’s argument that the FDA should be overhauled and run clinical studies itself, as happens with the equivalent agency in Italy:
“The Italian FDA collects money from every drug company that sells drugs in Italy, pools that, and funds drug trials. They fund the sort of head-to-head drug comparisons that companies don’t like to fund. And they have independent people peer-reviewing the trials. It’s a great model,” [Lisa Bero] said.”
A comment in response to Pure Pedantry, a blog by a PhD/MD student at Mount Sinai School of Medicine, also called for new ethical rules for the publication of drug company results.
Some of the other work that Lisa Bero and her colleagues have done to raise awareness about faulty reporting of clinical trials and conflicts of interest can be found in the Loxosceles science blog, which summarizes her talk at the ScienceWriters2008 meeting. This makes reference to a meta-analysis that shows that peer-review is not necessarily a guarantee against bias – a drug company funded study is four times more likely to turn up a result favorable to its own product than is an independent study.
This paper follows research published in PLoS Medicine in September by Ida Sim and colleagues that also investigated the publication status and publication bias of trials submitted to the FDA for a wide variety of approved drugs.