Clinicians and patients are eager to know if antidepressants are effective and journalists recognize a hot topic when they see one. Science and health journalists in the media play a crucial role in communicating scientific findings to professional and lay audiences, but their goals in doing so are not limited to conveying best-evidence assessments of often complex and contradictory findings, and also include attracting readership to themselves and their media outlets.
As in other journalism, there’s often a reason to suspect that exaggeration and distortion gets introduced into science reporting in order to generate buzz. The goal of preserving necessary complexity is often no match for opportunities for publicity-grabbing hype. Recently, a producer of a TV news story about antidepressants told a well-known psychiatrist-researcher colleague of mine “You really make sense, but what you say would so complicate what I have to say,” and left him out of the story.
Scientists frequently become frustrated when they see their messages twisted in media reports, but we need to realize that journalists don’t work for us, they work for the media and themselves. Some scientists know that and play to the media, spinning interpretation of their finding in the scientific literature so that they are media-ready from the get go. A recent study demonstrated that apparent media distortion of science often starts with distorted scientific articles, and particularly those with hyped and exaggerated abstracts.
Psychiatrist Adrian Preda’s guest blog post for Mind the Brain hits on the all too familiar theme of supposedly dispassionate and objective science authors pitching exaggerated claims about antidepressants (back and forth) to responsive journalists in ways that leave clinicians and laypersons confused and ambivalent about important decisions concerning medical treatment. Clinicians are left contending not only with their own confusion, but that of patients who come back to question past decisions.
I have blogged a number of times elsewhere about a psychologist mentioned in Dr. Preda’s blog post, Irving Kirsch. I noted how even a seasoned and award-winning television media person was left bewildered interviewing him for CBS Sunday Morning News when he claimed that antidepressants did not significantly improve patients’ depression better than a placebo pill. In the lively exchange that followed, I further showed that if we were to adopt his arbitrary criterion for ‘significantly better,’ we would have to accept not only that antidepressants do not work, but that psychotherapy does not either, and that many medical treatments widely seen as effective actually are not. A recent detailed dissection of the statistical malpractice in Kirsch’s analyses probably will not dampen his repetition of his claims or their echoing in the media.
And now in a case of what Yogi Berra would call “deja vu all over again” Adrian Preda reports on another episode of conflicting interpretations of the data concerning the efficacy of antidepressants getting further confused and selectively reported in the media. To paraphrase his quote from psychiatrist Michael Thase, I’m sure there’s no last word here. –Jim Coyne
The Antidepressant Wars, a Sequel: How the Media Distort Findings and Do Harm to Patients
Guest blog post by Adrian Preda MD
Central to the perspective I present in this blog post is my work supervising psychiatric residents and medical students at a university-based psychiatry clinic where our patient population includes a good number of adults suffering from mild to moderate depression.
In 2010, the publication by JAMA of a single-study challenged and upended a major assumption that had guided clinical work like ours for over three decades (Barrett 2001; Qaseem 2008). This was the widely covered meta-analysis of antidepressant (AD) trials conducted by Fournier and colleagues(2010), which drew the far reaching conclusion that ADs show significant response in very severely depressed patients, but are not more effective than taking a placebo in less severe cases.
Fournier was not the first study that took aim at the foundation of treatment guidelines for depression, which in essence recommend treating depression with antidepressants. In 2008 Kirsch et al. meta-analysis of clinical trial data submitted to the Food and Drug Administration ended with a rather strongly worded conclusion:
“Drug–placebo differences in antidepressant efficacy increase as a function of baseline severity, but are relatively small even for severely depressed patients” (emphasis added). (Kirsch et al., 2008)
After reading their findings a neutral conclusion for Kirsch et al. would be that
- ADs are statistically better than placebo
- Response correlates with patients’ severity of symptoms.
Not an earth shattering conclusion by any means as both results were already common knowledge for anyone who started prescribing ADs since 2002, the date when Kahn et al. published their 45 studies based meta-analysis of FDA submitted AD trial data. Their conclusion?
“The magnitude of symptom reduction was significantly related to [..] initial depression […] scores; the higher the […] initial […] score, the larger the change.” (Kahn et al. 2002)
Therefore, one can look at the Kirsch (2008) study findings as a replication of earlier findings, a continuation of a line of knowledge that has already been established. Which is most times the way scientific knowledge expands. Given this, one would be hard pressed to understand how a study that essentially replicated prior positive findings would become the poster child for the anti-antidepressant movement that followed. But that is exactly what happened.
Interestingly, Kahn et al. (2002) was not cited by Kirsch et al. (2008), in itself a remarkable oversight considering the similarities between the two studies. But I found even more troubling that instead of conservatively explaining their findings and providing as much of a neutral and tentative explanation as possible — the validated scientific communication tradition — Kirsch et al. appeared to formulate their conclusion from a position of commitment to an anti-antidepressant view:
“The relationship between initial severity and antidepressant efficacy is attributable to decreased responsiveness to placebo [even] among very severely depressed patients, rather than to increased responsiveness to medication.” (Kirsch et al., 2008)
And that strongly worded conclusion made the Kirsch study an almost overnight media hit. Front-page newspaper and radio coverage followed, criticism was dismissed (Horder 2011).
To this date, the Kirsch study remains one of the most popular papers on the PLoS Medicine website, as reflected in the following metrics: 282,219 views, 631 citations, 300 academic bookmarks, and 404 social share (data as of December 20th, 2012). A number of critical commentaries followed. Some directly criticized Kirsch et al. (2008) for methodology or overstated conclusions (Kelly, 2008; Khan and Khan, 2008; McAllister-Williams, 2008a, 2008b; Moller, 2008; Nutt and Malizia 2008; Parker, 2009; Turner and Rosenthal, 2008). More interestingly, a few who decided to re-analyze Kirsch’s data found they could not replicate Kirsch et al. pessimistic view on AD’s efficacy (Fountoulakis , 2011; Horder et al., 2011). For unclear reasons these subsequent reports aimed at reestablishing the ADs respectability got much less media attention than Kirsch’s 2008 original.
“Déjà vu All Over Again”
In this context when Fournier at al. came along in 2010 I thought I had a déjà vu. Not as much in terms of the study’s conclusions but rather in terms of the emotional intensity and dramatic flavor with which it was greeted by the mass media. I first heard about it on NPR, and this surprised me as I usually get the studies I am interested in before the media does.
Over the next couple of days headlines such as these appeared in print and online media around the world:
- The NY Times: “Popular Drugs May Help Only Severe Depression”(Carey B, 2010)
- From the LA Times: “Antidepressant medications probably provide little or no benefit to people with mild or moderate depression” (Roan S, 2010)
Immediately following this media hoopla, I found that my students – a new generation who have not been part of the Kirsch antidepressants wars – began to routinely question the wisdom of continuing or starting antidepressant treatment for our patients suffering from mild or moderate depression.
And it did not take long for our patients themselves to express their doubts about the efficacy of antidepressants — even for severe depression.
I was troubled at the time by the unquestioning coverage of Fournier et al which inferred that this single study was in fact “settled science” on the subject of antidepressants when it was not; and by the inattention given (in either the professional literature or popular press) to either the complexities or long history of debate (as discussed above) or at least the serious flaws in the study’s methodology – as I’ve summarized below.
Two years later I’m equally concerned about the lack of media coverage given to a 2012 publication, also by JAMA, of a study by Gibbons and colleagues (2012) which, history aside, refutes Fournier’s claim that antidepressants are not more effective than placebo for mild to moderate depression. Similar to Fournier et al. (2010) Gibbons et al.’s (2012) findings are based on individual patient data and include longitudinal measurement which makes its conclusions a strong counterpoint to those of Fournier et al. (2010).
Among the points I now make to my students when questions arise about antidepressant efficacy as a result of the meta-analysis conducted by Fournier et al, are the following:
- The individual patient-level data approach used by Fournier et al represented an improvement over standard meta-analyses; however their results were based on only 6 studies that met their criteria from more than 200 relevant studies
- Reducing 2164 citations to 6 is hardly representative, especially when the 6 analyzed studies represent only two medications: paroxetine and imipramine, the latter not recommended for first line treatment of depressive disorders
- Furthermore, of the 6 studies, 5 specifically excluded with very mild depression making the authors’ conclusions about lack of separation of ADs from placebo for mild depression weak.
Exclusion Criteria Raise Major Questions
The strength of a meta-analysis is based on applying a solid statistical approach to all studies meeting a set of relevant inclusion/exclusion criteria, and in this case it appeared that the authors excluded too many relevant studies. Specifically, 228 studies were excluded based upon their exclusionary “placebo washout lead-in” requirement (a requirement that all study participants get a placebo to start with and only those who do not respond to the placebo continue in the study). The placebo washout/lead-in represents a common historical design used in antidepressant trials with the intent of excluding patients who do not demonstrate symptom stability thus are not likely to benefit from a truly effective AD. Fournier et al. (2010) acknowledge that “it is not clear that placebo washouts actually enhance the statistical power of antidepressant medication/placebo comparisons” nevertheless they proposed that in order to evaluate the rates of “true placebo response” one should exclude all studies using a placebo wash-out/ lead-in design.
While it is true that a placebo washout might limit accurate estimates of placebo-response and might not improve the probability of an AD being more effective that a placebo, this design for studies of depression would not affect the validity of an active AD – placebo separation, were one to be found. The exclusion of washout studies was especially problematic precisely because this represents a common design for AD clinical trials, meaning that numerous relevant studies will be excluded. In other words Fournier et al. imposed a seemingly arbitrary (i.e. not evidence based) exclusionary criterion that effectively filtered out the majority of the relevant studies. This is a very bright red flag and potential source of bias, which greatly limits the validity of the authors’ conclusions. Assuming these easily excluded studies were otherwise methodologically sound, the number of study investigators contacted would have increased from 23 to 251; and likely significantly more than 6 would have contributed to the final analysis.
Considering the potentially grave implications of either mental health providers or patients accepting the headlines generated by widespread publication of these results at face value, the study’s methodological weaknesses –which were not treated in any depth by comments accepted for publication by JAMA – warrant further critical review.
Overlooked and Highly Relevant Research
Likely because it received dramatically less coverage, far fewer of my students are aware of the 2012 study by Gibbons et al (2012) who, after reviewing 43 fluoxetine and venlafaxine trials, concluded that, contrary to the Fournier at al. (2010) findings, these two antidepressants are in fact efficacious for major depressive disorder in all age groups, regardless of the depression severity at baseline.
As noted, Gibbons et al. (2012), as Fournier et al. (2010), also used patient-level data – making the point against Fournier el al. even more significant. In addition, if you compare Gibbons et al. (2012) final set of 43 studies with a meta-analysis population of 4303 patients in the fluoxetine trials and 4882 patients in the venlafaxine trials (in total more 9000 patients) to the Fournier et al. (2010) final set of 6 studies (3 paroxetine and 3 imipramine trials) with a total of 718 patients, Gibbons et al. (2012) significantly larger number of studies makes for a more believable conclusion.
Both studies are limited in that they focused on only 2 ADs: paroxetine and imipramine for Fournier et al. (2010) versus fluoxetine and venlafaxine for Gibbons et al. (2012). At the same time Gibbons et al. (2012) used an all-inclusive set of studies, whereas, as noted above, the Fournier et al. (2010) study used a highly selective group of studies. There are also important differences in data analytic methods that could explain the differences in results. For example, Gibbons et al. (2012) defined severity differently than Fournier et al. (2010).
To expert eyes, the main effects for the drug versus placebo differences can be actually seen as similar in the two data sets. And that is the very reason for engaging in this debate.
Which study is more convincing?
The Gibbons study reminds us that it is our duty as physicians and society at large to carefully screen and aggressively treat depression, including with medications if so recommended. The Fournier study makes us aware that there might be more to the story of AD response than a straightforward active ingredient effect.
We can all speculate about why the Gibbons study received so much less media coverage than did Fournier and colleagues.
In the antidepressant wars, we have seen the pendulum’s full swing from the early nineties when Elisabeth Wurtzel’s “Prozac Nation” was thrilled to be “Listening to Prozac” with Peter Kramer, and into the early millennium years when Healy’s tongue in cheek advice was to “Let Them Eat Prozac”. By the time Carl Elloit’s “Prozac as a Way of Life” hit the stands in 2003, some thought we were at the end of an era. But ADs came back strong, only to engender renewed debate and, as argued above, uneven and thus inaccurate media coverage in the current decade.
Unintended Consequences of an Unevenly Covered Debate
As my esteemed colleague Michael Thase adeptly put it to me, “There is no ‘last word’ in the science of this debate.” He is undoubtedly correct. And, as a physician, I find relief in the fact that we continue to question engrained assumption and are reluctant to accept there is such a thing as a last word or simple explanation when it comes to complex issues. Depression, with its multidimensional tentacles equally anchored in nature and nurture will never be a good subject for simple explanations.
But, again, as a physician I am very concerned about major unintended consequences of uneven coverage of the competing major findings discussed above. Specifically, I fear that clinically depressed members of the public at large will refuse a likely efficacious treatment option. And while all may be well if that depressed patient makes the informed alternative choice of starting treatment with cognitive behavioral therapy (CBT), a validated form of therapy for depression that compares well with SSRIs for mild or moderate depression, all is certainly NOT well if the patient’s decision not to accept treatment with antidepressants is based primarily on media delivered misinformation.
Given the stigma against acknowledging or treating a mental illness with a psychotropic medication, the media saturation given to one study only worsens an already difficult situation for many patients who fear the personal and social consequences of admitting their illness and seeking treatment.
In closing: my hope is that members of the media who cover this debate will realize that “first do no harm” is not only the duty of physicians; it is also the responsibility of anyone trusted with giving health information to the public at large.
Acknowledgements: I would like to thank Lawrence Faziola and Steven Potkin for critically discussing Fournier et al. and Michael Thase for his critical read of the draft to this article.
Adrian Preda MD is a psychiatrist and Health Sciences Professor of Psychiatry and Human Behavior at University of California Irvine School of Medicine. He received his residency training at Yale, and has been a faculty member at Yale and UT Southwestern, prior to joining U.C. Irvine.
Barrett JE, Williams JW Jr, Oxman TE; et al. (2001) Treatment of dysthymia and minor depression in primary care: a randomized trial in patients aged 18 to 59 years. J Fam Pract. 50(5):405-412.
Carey B (2010) Popular Drugs May Help Only Severe Depression. New York Times, January 5, 2010
Fournier JC, DeRubeis RJ, Hollon SD; et al. (2010) Antidepressant Drug Effects and Depression Severity A Patient-Level Meta-analysis. JAMA303(1):47-53.
Fountoulakis KN, Möller HJ (2011) Efficacy of antidepressants: a re-analysis and re-interpretation of the Kirsch data. Int J Neuropsychopharmacol. 14(3):405-12. Epub 2010 Aug 27.
Gibbons RD, Hur K, Brown CH, Davis JM, Mann JJ (2012) Benefits from antidepressants: synthesis of 6-week patient-level outcomes from double-blind placebo-controlled randomized trials of fluoxetine and venlafaxine.Arch Gen Psychiatry 69(6):572-9.
Horder J, Matthews P, Waldmann R. (2011) Placebo, prozac and PLoS: significant lessons for psychopharmacology. J Psychopharmacol. 25(10):1277-88.Epub 2010 Jun 22.
Kelly BD (2008) Do new-generation antidepressants work?. Ir Med J 101: 155–155.
Khan A, Leventhal RM, Khan SR, Brown WA (2002) Severity of depression and response to antidepressants and placebo: an analysis of the Food and Drug Administration database. J Clin Psychopharmacol 22: 40–45.
Khan A, Khan S (2008) Placebo response in depression: a perspective for clinical practice. Psychopharmacol Bull 41: 91–98.
Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, Johnson BT (2008) Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med 5: e45–e45.
McAllister-Williams RH (2008a) Do antidepressants work? A commentary on ‘Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration’ by Kirsch et al. Evid Based Ment Health 11: 66–68.
McAllister-Williams RH (2008b) Misinterpretation of randomized trial evidence: Do antidepressants work?. Br J Hosp Med (Lond) 69: 246–247.
Moller HJ (2001) Methodological aspects in the assessment of severity of depression by the Hamilton Depression Scale. Eur Arch Psychiatry Clin Neurosci 251(suppl 2): II13–20.
Moller HJ (2008) Isn’t the efficacy of antidepressants clinically relevant? A critical comment on the results of the metaanalysis by Kirsch et al. 2008. Eur Arch Psychiatry Clin Neurosci 258: 451–455.
Nutt DJ, Malizia A (2008) Why does the world have such a ‘down’ on antidepressants?. J Psychopharmacol 22: 223–226.
Qaseem A, Snow V, Denberg TD, Forciea MA; et al. (2008) Clinical Efficacy Assessment Subcommittee of American College of Physicians. Using second-generation antidepressants to treat depressive disorders: a clinical practice guideline from the American College of Physicians. Ann Intern Med. 149(10):725-33.
Parker G (2009) Antidepressants on trial: how valid is the evidence?. Br J Psychiatry 19: 1–3. Web of Science
Turner EH, Rosenthal R (2008) Efficacy of antidepressants. Br Med J 336: 516–517.
Roan S (2010) Study finds medication of little help to patients with mild, moderate depression. Los Angeles Times. January 06, 2010.
Adrian Preda, M.D., Health Sciences Professor, Department of Psychiatry and Human Behavior, University of California, Irvine School of Medicine firstname.lastname@example.org