On January 29th 2014, researchers from UCLA, NCI, and Consolidated Research, Inc. published an observational study in PLOS ONE detailing increased cardiovascular risks associated with men taking testosterone therapy. Only three days later, on January 31st, this study in combination with another, published in JAMA in November 2013 with similar findings, triggered a Food and Drug Administration (FDA) Drug Safety Communication and evaluation that is currently ongoing. Since then, lawsuits against testosterone drug makers have been filed in both the US and Canada, and the FDA has set up a joint advisory committee who will meet this fall to evaluate potential health risks associated with testosterone therapies to determine whether any regulatory measures are required.
Why all the commotion around one or two scientific articles? The authors of the PLOS ONE study report that men aged 65 years and older who filled a testosterone prescription had double the risk of heart attack within three months. The study is based on a sample of 55,000 men from a dataset of enrollees in a health market database during the years 2006-2010. For younger men with pre-existing heart disease, the increased risk was two- to three-fold higher within the same period.
For matters relating to public health, this study is both relevant and controversial due to the skyrocketing of testosterone prescriptions to men, an industry now valued at $2 billion USD. Unfortunately, testosterone is often prescribed without a blood test for the medical diagnosis that warrants its use (hypogonadism), as it is also claimed to have other health benefits, including improved sex drive, mood, energy levels, and anti-aging effects.
While it’s clear that this PLOS ONE article impacted the actions of regulatory agencies and issues pertaining to public health, we wondered whether this influence was reflected in the paper’s article-level metrics (ALMs). Or, as Roli Roberts asked in a recent PLOS Biologue post, “how do you know when you’ve published a great paper?” Or even a good paper?
As it turns out, this paper did have an interesting, atypical trajectory post-publication. Rather than gaining traction first in the media, riding a flurry of social media activity, and then eventually attracting the attention of health regulators, immediate action was taken by the FDA within days of publication. Concurrently, the article received an explosion of media coverage within a week post-publication, including news pieces in the New York Times: Well Blog, NPR: Health News, and The Wall Street Journal MarketWatch, among others, and was continuing to receive coverage in the BBC News and NPR as late as April 2014. All of this coverage was recorded by our media tracking tool, which can be found under “Related Content” here.
Is the influence of this article captured in its ALMs? If we glance at individual measures, the article had ~ 21,000 views in the first month of publication, and currently has well over 36,000 HTML page views, 17% of which have resulted in PDF downloads. If we compare this view number to that of all other PLOS ONE articles published in the same timeframe, we see that this article is ranked 12th most-viewed. Additionally, a PDF download percentage of 17% is roughly average for a PLOS ONE article, where a higher percentage may indicate increased scholarly usage relative to other, more mainstream types of usage. On social media, this article was shared well over 100 times, which is substantial but less than occurred for other articles published since, with a few papers currently at well over 1000 shares.
However, if we examine citations and post-publication commentary seven months since publication, the study has already accumulated 20 citations, more than any other article published in this timeframe. In addition to accumulating citations, the article has prompted researchers to call for additional, more definitive studies: an article published in The Lancet Diabetes and Endocrinology cited the PLOS ONE and JAMA articles and declared an urgent need for large clinical trials of testosterone. Last but not least, the article has stimulated significant post-publication commentary from readers who have analyzed and critiqued the methodology and characteristics of the cohort, spurring further discussion and research, and effectively moving post-publication peer review in a forward direction.
Therefore, the value of this article may not lie strictly in its views, shares, citations, or even saves, or any combination or calculation thereof, but instead in its open access visibility to the public, healthcare providers, federal agencies, and media outlets. Anyone, anywhere in the world with Internet access can read this article and its surrounding discussion and then decide for themselves how to respond and react, and that alone is worth more than any level of metrics combined.
Finkle WD, Greenland S, Ridgeway GK, Adams JL, Frasco MA, et al. (2014) Increased Risk of Non-Fatal Myocardial Infarction Following Testosterone Therapy Prescription in Men. PLoS ONE 9(1): e85805. doi:10.1371/journal.pone.0085805
Vigen R, O’Donnell CI, Barón AE, et al. Association of Testosterone Therapy With Mortality, Myocardial Infarction, and Stroke in Men With Low Testosterone Levels. JAMA. 2013;310(17):1829-1836. doi:10.1001/jama.2013.280386
Piszczek J, Mamdani M, Antoniou T, Juurlink DN, Gomes T (2014) The Impact of Drug Reimbursement Policy on Rates of Testosterone Replacement Therapy among Older Men. PLoS ONE 9(7): e98003. doi:10.1371/journal.pone.0098003
Rx image from Ano Lobb on Flickr.
Testosterone image from Benjah-bmm27.