Soothing psychotherapists’ brains with NeuroBalm

Promoters of Emotionally Focused Psychotherapy offer sciencey claims with undeclared conflicts of interest, cherry picked evidence, and bad science.

The temptation exists for researchers and clinicians to search for the strongest and most provocative version of their knowledge, which will create greatest publicity. The appeal is great; oversell and over-dramatize the result and attention will follow. — Jay Lebow, Editor, Family Process

amygdalaPity the poor therapists. They want to do the best for their clients. They are required to get CE credits for licensure and renewal. But how do they choose their CE courses? With workshop promoters hawking approved courses in thought field therapy and somatic experiencing therapies, therapists can understand that professional organizations’ approval is no guarantee that what they will learn is evidence- supported or that it will mostly help, rather than hurt their clients.

Worse, few therapists have the research background minimally necessary to interpret the sometimes wild claims made promoters of workshops. They are unprepared to evaluate impressively sciency claims that are being made for treatments. And what is more sciency than neuroscience?

Psychotherapy is an inherently uncertain, subjective process. Isolated in sessions with clients, therapists do not have ready ways to monitor what is going on with confidence and decide moment to moment if it is helpful. Even when psychotherapy is manualized, done by the book, there is lots of uncertainty as to what is to be done when, to whom, whether it is done effectively, and how to follow-up.

Neuroscience seems to hold the promise of reducing some of that uncertainty. Exploitative hucksters make lots of money from therapists and their clients with claims that they can use neuroscience to monitor and direct the process of psychotherapy with precision. The hucksters play on the belief that changes in neural functioning can somehow serve to get more at what is “really” going on in therapy, beyond and, if necessary, in sharp contradiction of what therapists observe and clients report.

Enter workshop promoter Susan Johnson.  As told in the New York Times, she claims her emotionally-focused therapy (EFT)

can help couples break out of patterns, “interrupting and dismantling these destructive sequences and then actively constructing a more emotionally open and receptive way of interacting.” She aims to transform relationships “using the megawatt power of the wired-in longing for contact and care that defines our species,” and offers various exercises to restore trust.

Wow! If we could only monitor that interrupting and dismantling and the megawatt power of the “wired-in longing” with neuroscience.

In this blog post I discuss an article in PLOS One in which psychotherapist Johnson teams up with neuroscientist Jim Coan to claim they can do just that.

Ultimately, our handholding paradigm has provided a unique opportunity to test some of the proposed mechanisms of social support in general, and EFT in particular, all at the level of brain function, in vivo.

It is a terrible article, starting with its undisclosed conflicts of interests: Johnson is using the article to promote her psychotherapy products. And when we get past that, the article is shamelessly blatant cherry-picked evidence and poor psychotherapy research. We can learn from it as such.

Click in text for video

Click in text for video

But wait, hold on! Think of me like the greeter at the local Kanuka_BalmMacy’s department store who sprays you with free cologne or maybe rubs your hands with a soothing balm. Before we get into discussing the article, you can get a free sample of the Neurobalm right here that is being used to promote this psychotherapy product. See, no, feel for yourself. This is best appreciated wearing high-quality earphones to do the wonderful soundtrack justice.

Disclaimer: As you can already tell, I find this article outrageous and I am just getting warmed up in explaining how and why.  I am a PLOS One Academic Editor and I have gone on record insisting that promoters of psychotherapy be held to the same standards as the pharmaceutical companies in having to disclose apparent conflicts of interest. And now I have encountered an undisclosed conflict in the very journal where I work for free to provide a small bit of the oversight of the quality and integrity of what readers find there.

Oversight of conflicts of interest is far from perfect, especially when it depends on author disclosure. And oversight of the 24,000 articles published in PLOS One last year cannot be expected to be perfect.  But PLOS One has numerous tools to be self-correcting, especially when faced with undisclosed conflicts of interest. Unlike the journals Prevention Science and Clinical Psychology Review that I have been recently complaining about, PLOS One asks every author about potential conflicts of interest and every article published in PLOS One has an explicit declaration. And unlike these other two journals, PLOS One has explicit, orderly procedures for responding to apparent non-disclosures. An editor like myself, just like any reader, can make a complaint, and PLOS One will evaluate whether an inquiry to authors is necessary in order to decide what further action to take.

disclaimerThe opinions I am going to express here are my own, and not necessarily those of the journal or other members of the editorial board. Thankfully, at Mind the Brain, bloggers are free to speak out for themselves without censorship or even approval from the sponsoring journal. Remember what happened at Psychology Today and how I came to blog here.

The full text of the open access article is available here.


Social relationships are tightly linked to health and well-being. Recent work suggests that social relationships can even serve vital emotion regulation functions by minimizing threat-related neural activity. But relationship distress remains a significant public health problem in North America and elsewhere. A promising approach to helping couples both resolve relationship distress and nurture effective interpersonal functioning is Emotionally Focused Therapy for couples (EFT), a manualized, empirically supported therapy that is strongly focused on repairing adult attachment bonds. We sought to examine a neural index of social emotion regulation as a potential mediator of the effects of EFT. Specifically, we examined the effectiveness of EFT for modifying the social regulation of neural threat responding using an fMRI-based handholding procedure. Results suggest that EFT altered the brain’s representation of threat cues in the presence of a romantic partner. EFT-related changes during stranger handholding were also observed, but stranger effects were dependent upon self-reported relationship quality. EFT also appeared to increase threat-related brain activity in regions associated with self-regulation during the nohandholding condition. These findings provide a critical window into the regulatory mechanisms of close relationships in general and EFT in particular.

Before co-authoring this PLOS One article with Susan Johnson, Jim Coan published a closely related 2006 study in Psychological Science. Coan received lots of press coverage, even before the article was available on the Internet. The blogger Neurocritic critiqued the press coverage and then followed up with a blog post critiquing the present PLOS One article.

Neurocritic provides some generally useful wisdom concerning interpreting statements about neural imaging, psychotherapy, and relationships. I think most neuroscientists would agree with him. If you are a therapist, you might want to bookmark his blog post for future reference when you feel slathered with Neurobalm from psychotherapy workshop gurus.

An extended excerpt from Neurocritic

Can neuroscience illuminate the nature of human relationships? Or does it primarily serve as a prop to sell self-help books? The neurorelationship cottage industry touts the importance of brain research for understanding romance and commitment. But any knowledge of the brain is completely unnecessary for issuing take-home messages like tips on maintaining a successful marriage.

In an analogous fashion, we can ask whether successful psychotherapy depends on having detailed knowledge of the mechanisms of “neuroplasticity” (a vague and clichéd term). Obviously not (or else everyone’s been doing it wrong). Of course the brain changes after 12 sessions of psychotherapy, just as it changes after watching 12 episodes of Dexter. The important question is whether knowing the pattern of neural changes (via fMRI) can inform how treatment is administered. Or whether pre-treatment neuroimaging can predict which therapy will be the most effective.

However, neuroimaging studies of psychotherapy that have absolutely no control conditions are of limited usefulness. We don’t know what sort of changes would have happened over an equivalent amount of time with no intervention. More importantly, we don’t know whether the specific therapy under consideration is better than another form of psychotherapy, or better than going bowling once a week.

Problems start  with the article’s title

Soothing the threatened brain: Leveraging contact comfort with Emotionally Focused Therapy.

This title titillates the unwary but triggers an alert among even open minded skeptics.

Some of you may recall that in the tips I gave for writing titles in the Colon Theory of Titles. I suggested that if you reserve one side of the colon in a title for keywords, you might use the other side to have a little fun attracting interest in your article.

Coyne, J. C., & van Sonderen, E. (2012). The Hospital Anxiety and Depression Scale (HADS) is dead, but like Elvis, there will still be citings. Journal of Psychosomatic Research, 73(1), 77-78.

Or the more outrageous

Krauth, S. J., Coulibaly, J. T., Knopp, S., Traoré, M., N’Goran, E. K., & Utzinger, J. (2012). An in-depth analysis of a piece of shit: distribution of Schistosoma mansoni and hookworm eggs in human stool. PLOS Neglected Tropical Diseases, 6(12), e1969.

Fair fun. But the problem with Soothing the Threatened Brain is that many of the therapists think there is something more profound about ‘soothing the brain’, rather than soothing the wife or her heart or her emotions. And this target audience is all too ready to believe that there is something special about promoters of emotion focused therapy claiming it soothes the threatened brain. EFT is better than other marital therapies because it works on the wife’s brain, not just a couple. Other therapies only do relationships or wives but EFT does brains.

If you think I am being too tough on therapists, you can do an informal experiment. Strike up conversations with a few therapists about how they understand the abstract and title of this article or the dramatic video is based on the study. I tried this, and though some expressed some skepticism, they really did not feel competent to argue with a peer-reviewed article or a video of a fMRI assessment.

undeclaredAppearance of conflict of interest

Competing interests: The authors have declared that no competing interests exist.

Funding. This research was supported in part by the International Centre for Excellence in Emotionally Focused Therapy (ICEEFT), a not-for-profit corporation whose mission includes the scientific evaluation of EFT. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Additional funding was provided by a National Institute of Mental Health grant, Award Number R01MH080725, awarded to JAC. No additional external funding received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

This does not ring true. The website of the International Center for Excellence and Emotion Focused Therapy lists Susan Johnson as founder and director. Not only did Susan Johnson design the study and interpret the data, she provided supervision of the therapy and somehow decided the particular time point at which women who had received the intervention would get fMRI assessment. As psychotherapy research, this is bizarre and breaks down any benefits of experimental control — the investigator with strong allegiance to her treatment gets to pick when outcome assessments are done rather than having preset times of assessment.

The website for Johnson’s for-profit Ottawa Couples and Family Institute indicates that it shares the same physical space and administrative staff as ICEEFT. The nonprofit corporation serves a number of marketing functions, including maintaining a referral list of therapists who have completed sufficient trainings to obtain the certification, as well as granting permission to otherwise unqualified persons to participate in workshops and get certification so they can practice in the community, often without licensure. Depending on the country or state, people who attend trainings can offer EFT for a fee without having a license or any regulation as long as they do not adopt a title that is regulated and licensed in that particular jurisdiction. This can be a matter of calling themselves a coach or counselor, depending on the jurisdiction.

This is a rather standard arrangement in the therapy training business, whereby profit-making activities are from ostensibly nonprofit certification that extends the market for trainings.

If readers were informed of financial interests at stake….

If a candid conflict of interest statement had been provided, readers would have been more prepared to independently and skeptically evaluate claims starting in introduction and colorful and on scientific language throughout.

For instance, the authors declare in the introduction

Early research suggested that EFT was superior to behavioral marital therapy [20], and a more recent meta-analysis [21] concluded that 70–73% of couples who undergo EFT are no longer relationally distressed at the end of therapy – at an average effect size of d= 1.3.

The evidence of superiority [20] refers to a 1986 study with 15 couples each assigned to EFT or behavioral marital therapy. It was a small underpowered study that can be discounted by its high risk of bias, including the developers testing their own therapy.

Let us get real. Accumulated psychotherapy studies suggest that it is quite unrealistic to expect that a comparison of 15 couples receiving a particular therapy versus 15 couples who were on a waiting list will yield a significant finding. There are also lots of studies suggesting only modest differences between credible, active, structured therapies like EFT versus behavioral marital therapy. It is highly unlikely that such findings would be obtained by honest and transparent reporting of well-done psychotherapy research by anyone without a dog in the fight.

The “more recent meta-analysis [21]” refers to a 1999 poor quality review and meta-analysis also conducted by developers of EFT.

The meta-analysis is worth a look. You can click on the table to the right and see the 7 table EFT meta-analysis 1999-1studies included. All were done by developers of EFT or as a dissertation under the supervision. Three studies are nonrandomized trials, one with only seven couples. All of the randomized trials have 16 or fewer couples assigned to EFT,  and so have less than 50% probability of detecting a positive result if it is present. All of the articles identified as positive studies.

Essentially this is a poor quality meta-analysis of what should have been left as pilot studies conducted by promoters of a therapy in their own lab. The meta-analysis lacks many of the formal features of meta-analyses including forest plots and assessments of risk of bias. The overall effect size of 1.31 is improbably high in the failsafe N of 49 studies being unpublished to unseat a positive evaluation EFT highlights the absurdity of invoking that statistic. If we took failsafe N seriously, we would have to accept that there would have to be almost as many unpublished null trials as there are couples in the published studies.

It is instructive to compare the assessment of the EFT from its developers to a more detached consideration about the same time by a group organized by American Psychological Association to evaluate the evidence-supported status of psychotherapies.

The APA group tell you things that somehow get missed in this review of EFT in PLOS One.

It is important to note that the interventions were restricted to moderately distressed couples because the investigators were concerned that EFT might not be optimal for extremely distressed couples.

Promoters of a psychotherapy rarely lose a comparison between the intervention they are rooting for and a rival comparison-control, particularly in a grossly underpowered study. However, that is exactly what occurred in one of the studies included in the meta-analysis were EFT was bested by strategic therapy in follow-up. This led the APA group to declare systemic therapy “possibly efficacious.” I doubt this kind of upset has ever happened in formal evaluations of psychotherapy research. Of course, the APA group’s rules are kind of loopy and I would not give this evaluation too much credence. Nonetheless, the APA group goes on:

This difference between treatments resulted from the couples in the EFT treatment experiencing significant relapse during the follow-up period. The investigators noted that couples in this study were much more distressed than couples in the Johnson and Greenberg study, which might account for the differences seen in the two studies at follow-up. They cautioned that, with severely distressed couples, time-limited EFT might not be powerful enough to create sufficient intimacy to maintain posttest gains.

Compare this to what Susan Johnson says in the PLOS article:

 Moreover, EFT treatment gains realized among distressed couples at high risk for relapse are stable over two- and three- year assessment periods [22,23]. *

Art Garfunkel’s Mr Shuck ‘N Jive

Art Garfunkel’s Mr Shuck ‘N Jive

The discrepancy can be explained by picking and choosing particular timepoints for particular tiny psychotherapy follow-up studies with highly selected, on representative patients.  Come on Susan, you’re shucking us. This has little resemblance to finding best evidence, you are just finding evidence to sell your psychotherapy.

The APA group also noted some differences in the outcomes of a waitlist control group in behavioral marital therapy conducted by its originator, Neil Jacobson:

Whereas 50% of James’s [an EFT study done by a dissertation student] waiting list couples improved without treatment, the waiting list couples in the BMT studies reviewed by Jacobson et al. showed an improvement rate of only 13.5%.

So, even the waitlist control groups do better in the EFT versus BMT studies.

Johnson continues her overview of the literature in the PLOS One introduction.

EFT has also been successfully applied to couples in which one or both partners are coping with a history of childhood sexual abuse [28,29], major depression [30,31], and even breast cancer [32].

You are shucking us again, Susan. What constitutes being “successfully applied”? These are not randomized controlled studies. For instance, the application to breast cancer involved only to patients. You are hardly in a position to crow about this. Shame on you.

When I read an introduction to a scientific article, I expect a much more nuanced, balanced consideration of the existing literature in a way that leads up to the research question of a particular study. What we get in this introduction in no way resembles us. Rather, an author with undeclared conflicts of interest is shamelessly hawking her psychotherapy product.

But stay tuned. In Part Two of this blog post I will offer a detailed critique of the methodology and interpretation of the actual study. It would be great if readers read the open access PLOS One article ahead of my next post and were prepared with their own interpretations and maybe even to dispute mind.

*The EFT literature and apparently what is said in workshops provide strong claims about outcomes that are echoed in the advertisements of therapists who get certified in EFT. For instance, the website of a Philadelphia-based therapist claims

EFT ad

Click to enlarge

This is either an exaggeration or outright fraud if it is supposed to represent the likelihood of a positive outcome of a couple coming to this therapy.

conflict of interest


Category: Conflict of interest, evidence-supported, mental health care, neuroscience, psychotherapy | Tagged , , , | 4 Comments

Repost: The Latest and Greatest in Treatment for PTSD: Magic Bullets and Cutting Edge Innovation

June is PTSD awareness month.  In light of this, I am reposting a blog I wrote about “The Latest and Greatest in Treatment for PTSD.”  If you are interested in knowing more about PTSD please check out the NCPTSD website.

Also, below are links to other blog posts I have written about PTSD and related topics:


The Latest and Greatest in Treatment for PTSD: Magic Bullets and Cutting Edge Innovation

I am frequently asked to talk about PTSD to professional audiences and, without 2012-04-05-ptsd1exception, always get a post talk question asking about my experience with some experimental intervention that someone read about somewhere in a newsmagazine or heard about from the T.V.

Internally, I always groan.

Having just spent 60-90 minutes pouring over carefully crafted PowerPoint slides that contain information about the evidence base for the treatments of PTSD and what best practices consist of, why I am always confronted with a zealous audience member who is obsessed with the new, the innovative, or the magic bullet?

In the interest of full disclosure, I have to share my viewpoint as being that of a health services researcher.  I approach PTSD treatment with a basic belief that we already have pretty good treatments, and the issues with getting better outcomes for PTSD lie more in how we implement those treatments, the limitations of the systems that provide care, massive issues of access to care (i.e. those who need care the most simply can’t access it for a myriad of reasons), and healthcare disparities (that an individual’s outcomes for PTSD are more likely linked to their zip code as opposed to their genes/neurotransmitters).

In short, I usually have a healthy skepticism toward the experimental or magic bullets type of treatments for PTSD, which often get a lot of media attention and can be very seductive to the brain of a researcher or clinician who spends their days trying to help individuals who live with PTSD.


Still, today I am curbing my skepticism and with much enthusiasm am writing about some of the hottest ideas for innovation in the treatment of PTSD.


Please note: MANY of these approaches are still considered EXPERIMENTAL, and I am listing them in no particular order of importance.

1. Mind – Body Practices for PTSD

Image Credit: Cornelius383

Mind Body practices are increasingly used to offer symptom reduction for PTSD.  Approaches such as Yoga, Tai Chi, Mindfulness Based Stress Reduction, Meditation, and Deep breathing are some examples.  There are about 16 rigorous studies that have been done to date, most of which have small sample sizes.  Whilst early findings suggest such practices can have a beneficial impact on symptoms like intrusive memories, avoidance, and increased emotional arousal, there is insufficient evidence to support their use as standalone treatments, though they can be recommended as an adjunctive treatment.


2. Cervical Sympathetic Blockade and Stellate Ganglion Block for PTSD 

In 2008, reports started to emerge about a minimally invasive manipulation of sympathetic nerve tissue in patients with PTSD that relieved their anxiety.  The procedure consisted of injecting a local anesthetic into sympathetic cervical nerve tissue at the C6 level and was apparently accompanied by immediate relief by the patient.  In 2012, a case series was reported where treatment resistant veterans with PTSD were given a stellate ganglion block and also a pre and post intervention CAPS. After the intervention, 5/9 of the patients experienced significant improvement; these benefits diminished over time and the benefits were not universal.  Controlled trials are currently underway to investigate this intervention further.


3. Virtual Reality Exposure Therapy

Virtual Reality exposure therapy utilizes real time computer graphics, body tracking devices, visual displays, and other sensory input devices to give the patient the experience that they are immersed in a virtual environment. It is an enhanced version of the imaginal exposure typically utilized as a part of trauma-focused psychotherapies. In 2001 an open clinical trial of Virtual Reality exposure therapy yielded promising results. It is currently being studied under controlled conditions.


4. D-Cycloserinemanypills

D-Cycloserine is a partial agonist of the NMDA receptor (a brain receptor that plays an essential role in learning and memory). It has been used to treat social phobia and panic disorder and to enhance the effects of psychological therapies for those disorders.  Preliminary data suggests it can be a useful adjunct in addition to evidence-based psychotherapies for patients living with severe PTSD.


5. Ketamine

Ketamine is a non-barbiturate anesthetic and antagonist at the NMDA receptor that is typically administered intravenously.  It has been used for years for patients with severe burns and it was, in this use, that its dissociative properties became apparent.  Retrospective studies show that those who received Ketamine after a traumatic event were less likely to develop PTSD.  It has been postulated that Ketamine may disrupt the process via which traumatic memories are laid down. A 2014 JAMA study reported on a RCT which demonstrated a rapid reduction in symptom severity following Ketamine infusion in patients with chronic PTSD.


6. Increasing the Intensity of Treatments

In an experimentation with packaging, British researchers compressed versions of trauma-focused psychotherapies for PTSD into a seven day intensive treatment.  This was found to work as well as treatment as usual, which is the same treatment delivered once a week, over 12 weeks.  Such an approach was postulated to be more efficient and convenient and was associated with faster improvement in symptoms and lower dropout rates.


7. Memantine colorful-pills

Memantine is a non competitive NMDA antagonist that is thought to protect the glutamergic destruction of neurons and hence prevent the hypothesized neurodegeneration in the hypothalamus, which contributes to the memory issues related to PTSD.  In a 2007 open label small trial, Memantine was found to be associated with some encouraging outcomes.  Double blind placebo controlled trials are pending.

Category: alternative medicine, Commentary, mental health care, Psychiatry, PTSD, research | Tagged , , , , | 1 Comment

Did the FDA antidepressants warnings increased suicidality?

A new study just published in the BMJ makes this very point.

bmj-logo-ogChanges in antidepressant use by young people and suicidal behavior after FDA warnings and media coverage: quasi-experimental study

The title makes two claims:

  1. That the  2003 black box  US Food and Drug Administration warning about a possible increased risk of suicidality with antidepressants lead to a change in antidepressant prescriptions.
  2. Further, that there were changes in suicidal behavior following the said US FDAwarning.

While the title does not indicate the direction of the changes, the study conclusions carry little ambiguity:

Safety warnings about antidepressants and widespread media coverage decreased antidepressant use, and there were simultaneous increases in suicide attempts among young people.

This is obviously a big claim, with far reaching implications spreading from the level of primary care physicians who might consider changing their antidepressant prescription practices to the level of policy makers deciding on guidelines about antidepressants approvals and reimbursement.

In this post we will discuss some of the study limitations which, for unclear reasons, seem to have been all but ignored by the over-excited welcome that study received in the mainstream media.

Our goal here is to not discuss subtle academic limitations but rather obvious limitations which could have been picked even by a casual yet critical reader of the paper. We will then cursorily survey the media presentations of the paper and assess the quality of their review in terms of balance and fair criticism.

Let’s start with some of the study most overt shortcomings:

  1. The title is not entirely informative. The study reviews US data however the title seems to indicate world-wide findings.
  2. The study uses a quasi-experimental design assessing changes in outcomes after the FDA  warnings, controlling for pre-existing trends. However the reliability of the controls is not entirely convincing.
  3. Not much is known about the relationship (if any) between the individuals entered in the study because they were prescribed antidepressants and the individuals who took psychotropic overdoses.
  4. The study uses psychotropic drug poising data as a proxy for suicide but is that a good proxi?
  5. The study does not address the possibility that suicide has been on the raise for reasons having nothing to do with antidepressant prescriptions (such as the recent years economic crisis).

Big media enthusiastic welcome


First the Washington Post (Dennis) reports:

As a result [of the FDA warnings] antidepressant prescriptions fell sharply for adolescents age 10 to 17 and for young adults age 18 to 29. At the same time, researchers found that the number of suicide attempts rose by more that 20 percent in adolescents and by more than a third in young adults.

Comment from a non-study affiliated expert? YES

How many limitations are discussed? NONE

According to NBC News (Raymond) :

New research finds the warning backfired, causing an increase in suicide attempts by teens and young adults. After the FDA advisories and final black box warning that was issued in October 2004 and the media coverage surrounding this issue, the use of antidepressants in young people dropped by up to 31 percent.

Comment from a non-study affiliated expert? NO

How many limitations are discussed? NONE

Reuters (Seaman) reports that:

Antidepressant use decreased by 31 percent among adolescents, about 24 percent among young adults and about 15 percent among adults after the warnings were issued. At the same time, there were increases in the number of adolescents and young adults receiving medical attention for overdosing on psychiatric medicines, which the authors say is an accurate way to measure suicide attempts. Those poisoning increased by about 22 percent among adolescents and about 34 percent among young adults after the warnings. That translates to two additional poisoning per 100,000 adolescents and four more poisoning per 1,000 young adults, the researchers write.

Comment from a non-study affiliated expert? YES

How many limitations are discussed? ONE (limitation #5)

According to USA TODAY (Painter):

Warnings that antidepressant medications might prompt suicidal thinking in some young people may have backfired, resulting in more suicide attempts, new research suggests.

Comment from a non-study affiliated expert? YES

How many limitations are discussed? NONE

Forbes (DiSalvo) reports:

Antidepressant use fell 31 percent among adolescents and 24 percent among young adults after the FDA warnings, according to the study. Suicide attempts increased by almost 22 percent among adolescents and 33 percent among young adults in the same time period. Suicide attempts tracked in the study were largely the result of drug overdoses.

Comment from a non-study affiliated expert? NO

How many limitations are discussed? NONE

NPR (Stein) also reports on the story:

Antidepressant use nationally fell 31 percent among adolescents and 24 percent among young adults, the researchers reported. Suicide attempts increased by almost 22 percent among adolescents and 33 percent among young adults.

Comment from a non-study affiliated expert? YES

How many limitations are discussed? ONE (limitation #5)

Finally, the Boston Globe (Freyer) concludes that

instead of declining as hoped, suicide attempts over the next six years showed a “small but meaningful” uptick among people ages 10 to 29, according to a study published Wednesday in the journal BMJ. That increase followed a substantial drop in the use of antidepressants.

Comment from a non-study affiliated expert? YES

How many limitations are discussed? NONE

A few observations


To summarize: big media’s portrayal of the situation is straightforward. The FDA black box warnings were followed buy an almost 30% decrease in antidepressant use and an almost 30% increase in suicide rates for adolescents and young adults.The conclusion, while not stated directly, is implicit: antidepressants decrease suicide in this population.

The issue is that this conclusion is at odds with what the study really found:

1. The decrease in antidepressant prescriptions is not absolute but relative to a projected forward prediction of increased antidepressant use over the studied period. All is good if the prediction is accurate – but there in no way of actually assessing how correct these sort of predictions really are.

2. There is no absolute increase in suicides either. The increase rate is not for suicide for for psychiatric drugs poisoning, which might be an entirely different beast than suicide. Even if accepting the authors’ decision to use psychotropics poisoning as proxy for suicide the reported increase is again not absolute but relative to a predicted trajectory of in fact unknown accuracy for psychotropics poisoning.

3. Or maybe the suicide rate was on the raise anyway for reasons that have nothing to do with the FDA warnings or antidepressant prescriptions.  As discussed above the economic crisis of the last few years might have independently contributed to this state of affairs.

Which leads us to an interesting conclusion

Somewhat paradoxically, it turns out that a study that tongue-in-cheek points to the media’s uncritical coverage of medical news as a possible contributor to a public health issue receives an almost universal and equally uncritical acclaim from the same media that it rightfully criticizes.

In a past analysis of medical news reporting I stated that

[...] my hope is that members of the media who cover [medical] debate[s] will realize that “first do no harm” is not only the duty of physicians; it is also the responsibility of anyone trusted with giving health information to the public at large.

More than 2 years later I found that this conclusion still stands.

For my Storify discussion of this story – I’ll see you here.


Category: antidepressants, depression, mental health care, Uncategorized | Tagged , | 4 Comments

The Vital Importance of Integrating Primary Care and Mental Health Care: How the VA Does It

For the better part of the last two decades I have practiced psychiatry in a variety of different American healthcare systems, and over these years I have, on numerous occasions, heard psychiatric services referred to in manner that imply (often subtly) that such services are not medical care.  These references come not only from patients, but nurses and doctors (including myself) too.


“Yes Mr. Jones you need to follow up with your regular medical doctor about that issue.”


“Dr. Jain, I went to see my medical doctor and he told me my blood pressure was high.”


Yes, I have been guilty of propagating this false dichotomy myself, and I too end up colluding with this societal misperception that somehow psychiatric care is NOT medical care, but something separate or distinct from other medical services. I think I did it because, on a day to day basis, when I am busy in clinic it is easier to collude than to get into a debate about semantics.


Still, in today’s blog I want to highlight the fact that this artificial distinction between physical and mental health perpetuates much of the stigma and misperception that, we as a society, have toward mental illness. But, most importantly, I want to convey my belief that when mental and physical well-being are separated, healthcare becomes poor in quality.


When I did my medical school training in Great Britain, every single medical school student was required to complete a 3 month (minimum) rotation in psychiatry and, furthermore, psychiatry was one of the specialties that had to be passed, in clinical exams, at finals before your MD would be granted.  Why, might you ask, should a ENT surgeon/dermatologist/ER physician to-be  need to spend so much time training in psychiatry?

Firstly, the majority of British medical students become primary care doctors.  The system is set up that way, so there are relatively few spots for specialty training (e.g. cardiology or plastic surgery) and there is much more emphasis for medical students to become primary care doctors.  This is based on the premise that that is what the country needs so that is what medical schools should provide.

colorful-pillsIt’s well known that a significant percentage of patients seen in primary care have a mental illness/disorder as a primary problem.  In fact, most prescriptions for psychotropic medications are written in primary care, hence it is logical that every British medical graduate be well versed and adept in diagnosing, treating, and managing psychiatric disorders.

Secondly, if one looks at mental illness from a sheer epidemiological point of view, no physician can afford to not be well trained in the fundamentals of psychiatric practice.  I, as a psychiatrist, may or may not, in my career, treat a patient who also develops a testicular tumor, needs bypass surgery, or has a fractured hip, but epidemiologically speaking, my colleagues in urology, cardiology, and orthopedics WILL treat patients who have comorbid depression/anxiety or even severe mental illness such as Bipolar Disorder or Psychosis.  For this reason, it makes sense that these providers have some awareness or understanding of such disorders.

My experience with US healthcare is different; the business side of US medicine has a tendency to favor medical specialties that are procedure-based or that generate flash technologies that can be promoted and attract more market share.  Unfortunately, psychiatry often fares poorly when it comes to such business strategies.  Mental illness can be chronic, take time to treat, and there is rarely a quick fix or magic cure.  Moreover, mental illness can be associated with a downward drift, e.g. someone becomes psychotic, they lose their job, and then their health insurance etc. etc.


In the U.S. healthcare business the specialty of psychiatry is often not given a seat at the table.


I think this, in part, explains this nonsensical divide between “medical” and “psychiatric” that we often have in healthcare.  Of course, it is a fallacy. Mental health and physical health are intricately link on every level, from a cellular level to a more macro perspective of how human beings navigate their day to day life.  In my view, a sophisticated healthcare system should reflect this intricate relationship and integrate primary and psychiatric care, i.e. get rid of this false distinction or separation by physically placing both services in one clinic, side by side.

soldier-294476_640One American healthcare system has been a leader in integrating primary care and mental health care. That system is the Veterans Health Administration (VA). Unlike many other U.S healthcare systems (which place more emphasis on treating individuals), the VA is charged with taking care of a population, i.e. veterans. This mission guides where the VA places emphasis, so whatever the prevalent issues are for this population becomes the area where the VA will place emphasis and resources.


The VA aims to meet the needs of the population it is serving, and hence gives psychiatry a seat at the table.


With more than 1,000 outpatient clinics, the VA is the largest health care system in the United States, and it has a very clear sense of its population. Over the past 15 years, the VA has not only participated in some of the biggest studies of integrated care, but has made a commitment to provide patient-centered integrated care to its population.

For the last two years I have been in the role of medical director of the Primary Care-Behavioral Health Team at the VA Palo Alto Health Care System, and I spend most of my days right here at the interface between physical well-being and mental health.  Contrary to some of my previous experiences in healthcare, the last two years have taught me the following:


  • Our colleagues in primary care place very high value on psychiatric and psychological consultation from colleagues.


  • The clinical work is very rewarding and in many ways bypasses a lot of the frustrations we often feel as physicians working in fragmented health care systems.


  • As a specialist, your experience and knowledge can add enormous benefit in making health care more streamlined and patient-centric. There are many opportunities for psychiatrists to act as educators to both colleagues and patients about common misperceptions surrounding mental disorders and mental health care.


  • Being a consultant for and working closely with a team of professionals from various specialty backgrounds helps your own career development. It prevents you from getting rusty in areas of medicine other than psychiatry and keeps you on the cutting edge of how health care systems are evolving to meet the needs and demands of all stakeholders.


Integrated care is the way of the future, and I feel fortunate that I work in a system that is at the cutting edge of such innovation.


For more information about the integrated model at the VA Palo alto Healthcare System please follow this link.

Category: Commentary, mental health care, Psychiatry, Uncategorized | Tagged , , , , , | 4 Comments

Salvaging psychotherapy research: a manifesto

NOTE: Additional documentation and supplementary links and commentary are available at What We Need to Do to Redeem Psychotherapy Research.

Fueling Change in Psychotherapy Research with Greater Scrutiny and Public Accountability

John Ioannidis’s declarations that most positive findings are false and that most breakthrough discoveries are exaggerated or fail to replicate apply have as much to with psychotherapy as they do with biomedicine.

BadPharma-Dec2012alltrials_basic_logo2We should take a few tips from Ben Goldacre’s Bad Pharma and clean up the psychotherapy literature, paralleling what is being accomplished with pharmaceutical trials. Sure, much remains to be done to ensure the quality and transparency of drug studies and to get all of the data into public view. But the psychotherapy literature lags far behind and is far less reliable than the pharmaceutical literature.

As it now stands, the psychotherapy literature does not provide a dependable guide to policy makers, clinicians, and consumers attempting to assess the relative costs and benefits of choosing a particular therapy over others. If such stakeholders uncritically depend upon the psychotherapy literature to evaluate the evidence-supported status of treatments, they will be confused or misled.

Psychotherapy research is scandalously bad.

Many RCTs are underpowered, yet consistently obtain positive results by redefining the primary outcomes after results are known. The typical RCT is a small, methodologically flawed study conducted by investigators with strong allegiances to one of the treatments being evaluated. Which treatment is preferred by investigators is a better predictor of the outcome of the trial than the specific treatment being evaluated.

Many positive findings are created by spinning a combination of confirmatory bias, flexible rules of design, data analysis and reporting and significance chasing.

Many studies considered positive, including those that become highly cited, are basicallycherrypicking null trials for which results for the primary outcome are ignored, and post-hoc analysis of secondary outcomes and subgroup analyses are emphasized. Spin starts in abstracts and results that are reported there are almost always positive.

noceboThe bulk of psychotherapy RCTs involve comparisons between a single active treatment and an inactive or neutral control group such as wait list, no treatment, or “routine care” which is typically left undefined but in which exposure to treatment of adequate quality and intensity is not assured. At best these studies can tell us whether a treatment is better than doing nothing at all or than patients expecting treatment because they have enrolled in a trial and not getting it (nocebo).


Meta-analyses of psychotherapy often do not qualify conclusions by grade of evidence, ignore clinical and statistical heterogeneity, inadequately address investigator allegiance, downplay the domination by small trials with statistically improbable rates of positive findings, and ignore the extent to which positive effect sizes occur mainly in comparisons between active and inactive treatments.

Meta-analyses of psychotherapies are strongly biased toward concluding that treatments work, especially when conducted by those who have undeclared conflicts of interest, including developers and promoters of treatments that stand to gain financially from their branding as “evidence-supported.”

Overall, meta-analyses too heavily depend on underpowered, flawed studies conducted by investigators with strong allegiances to a particular treatment or to finding that psychotherapy is in general efficacious. When controls are introduced for risk of bias or investigator allegiance, affects greatly diminish or even disappear.

Conflicts of interest associated with authors having substantial financial benefits at stake are rarely disclosed in the studies that are reviewed or the meta-analyses themselves.

Designations of Treatments as Evidence-Supported

There are low thresholds for professional groups such as the American Psychological Association Division 12 or governmental organizations such as the US Substance Abuse and Mental Health Services Administration (SAMHSA) declaring treatments to be “evidence-supported.” Seldom are any treatments deemed ineffective or harmful by these groups.

Professional groups have conflicts of interest in wanting their members to be able to claim the treatments they practice are evidence-supported, while not wanting to restrict practitioner choice with labels of treatment as ineffective. Other sources of evaluation like SAMHSA depend heavily and uncritically on what promoters of particular psychotherapies submit in applications for “evidence supported status.”

"Everybody has won, and all must have prizes." Chapter 3 of Lewis Carroll's Alice's Adventures in Wonderland

“Everybody has won, and all must have prizes.” Chapter 3 of Lewis Carroll’s Alice’s Adventures in Wonderland

The possibility that there are no consistent differences among standardized, credible treatments across clinical problems is routinely ridiculed as the “dodo bird verdict” and rejected without systematic consideration of the literature for particular clinical problems. Yes, some studies find differences between two active, credible treatments in the absence of clear investigator allegiance, but these are unusual.

The Scam of Continuing Education Credit

thought field therapyRequirements that therapists obtain continuing education credit are intended to protect consumers from outdated, ineffective treatments. There is inadequate oversight of the scientific quality of what is offered. Bogus treatments are promoted with pseudoscientific claims. Organizations like the American Psychological Association (APA) prohibit groups of their members making statements protesting the quality of what is being offered and APA continues to allow CE for bogus and unproven treatments like thought field therapy and somatic experiencing.

Providing opportunities for continuing education credit is a lucrative business for both accrediting agencies and sponsors. In the competitive world of workshops and trainings, entertainment value trumps evidence. Training in delivery of manualized evidence-supported treatments has little appeal when alternative trainings emphasize patient testimonials and dramatic displays of sudden therapeutic gain in carefully edited videotapes, often with actors rather than actual patients.

Branding treatments as evidence supported is used to advertise workshops and trainings in which the particular crowd-pleasing interventions that are presented are not evidence supported.

Those who attend Acceptance and Commitment (ACT) workshops may see videotapes where the presenter cries with patients, recalling his own childhood.  They should ask themselves: “Entertaining, moving perhaps, but is this an evidence supported technique?

Psychotherapies with some support from evidence are advocated for conditions for which there is no evidence for their efficacy. What would be disallowed as “off label applications” for pharmaceuticals is routinely accepted in psychotherapy workshops.

We Know We Can Do Better

Psychotherapy research has achieved considerable sophistication in design, analyses, and strategies to compensate for missing data and elucidate mechanisms of change.

Psychotherapy research lags behind pharmaceutical research, but nonetheless hasCONSORT recommendations and requirements for trial preregistration, including specification of primary outcomes; completion of CONSORT checklists to ensure basic details of trials are reported; preregistration of meta-analyses and systematic reviews at sites like PROSPERO, as well as completion of the PRISMA checklist for adequacy of reporting of meta-analyses and systematic reviews.

nothing_to_declare1Declarations of conflicts of interest are rare and exposure of authors who routinely failed to disclose conflicts of interest is even rarer.

Departures from preregistered protocols in published reports of RCTs are common, and there is little checking of discrepancies in abstracts from results that were actually obtained or promised in preregistration by authors.  There is  inconsistent and incomplete adherence to these requirements. There is little likelihood that noncompliant authors will held accountable and  high incentive to report positive findings in order for a study is to be published in a prestigious journal such as the APA’s Journal of Consulting and Clinical Psychology (JCCP). Examining the abstracts of papers published in JCCP gives the impression that trials are almost always positive, even when seriously underpowered.

Psychotherapy research is conducted and evaluated within a club, a mutual admiration society in which members are careful not to disparage others’ results or enforce standards that they themselves might want relaxed when it comes to publishing their own research. There are rivalries between tribes like psychodynamic therapy and cognitive behavior therapy, but suppression of criticism within the tribes and in strenuous efforts to create the appearance that members of the tribes only do what works.

Reform from Without

Journals and their editors have often resisted changes such as adoption of CONSORT, structured abstracts, and preregistration of trials. The Communications and Publications Baord of the American Psychological Association made APA one of the last major holdout publishers to endorse CONSORT and initially provided an escape clause that CONSORT only applied to articles explicitly labeled as a randomized trial. The board also blocked a push by the Editor of Health Psychology for structured abstracts that reliably reported details needed to evaluate what had actually been done in trials and the results were obtained. In both instances, the committee was most concerned about the implications for the major outlet for clinical trials among its journals, Journal of Consulting and Clinical Psychology.

Although generally not an outlet for psychotherapy trials, the journals of the Associationvagal tone for Psychological Science (APS) show signs of even being worse offenders in terms of ignoring standards and commitment to confirmatory bias. For instance, it takes a reader a great deal of probing to discover that a high-profile paper of Barbara Fredrickson in Psychological Science was actually a randomized trial and further detective work to discover that it was a null trial. There is no sign that a CONSORT checklist was ever filed the study. And despite Frederickson using the spun Psychological Science trial report to promote her workshops, there is no conflict of interest declared.

The new APS Clinical Psychological Science show signs of even more selective publication and confirmatory bias than APA journals, producing newsworthy articles, to the exclusion of null and modest findings. There will undoubtedly be a struggle between APS and APA clinical journals for top position in the hierarchy publishing only papers that that are attention grabbing, even if flawed, while leaving to other journals that are considered less prestigious, the  publishing of negative trials and failed replications.

If there is to be reform, pressures must come from outside the field of psychotherapy, from those without vested interest in promoting particular treatments or the treatments offered by members of professional organizations. Pressures must come from skeptical external review by consumers and policymakers equipped to understand the games that psychotherapy researchers play in creating the appearance that all treatments work, but the dodo bird is dead.

Specific journals are reluctant to publish criticism of their publishing practices.  If we at first cannot gain publication in the offending journals of our concerns, we can rely on blogs and Twitter to call out editors and demand explanations of lapses in peer review and upholding of quality.

We need to raise stakeholders’ levels of skepticism, disseminate critical appraisal skills widely and provide for their application in evaluating exaggerated claim and methodological flaws in articles published in prestigious, high impact journals. Bad science in the evaluation of psychotherapy must be recognized as the current norm, not an anomaly.

We could get far by enforcing rules that we already have.

We need to continually expose journals’ failures to enforce rules about preregistration, disclosure of conflicts of interest, and discrepancies between published clinical trials and their preregistration.

There are too many blatant examples of investigators failing to deliver what they promised in the preregistration, registering after trials have started to accrue patients, and reviewers apparently not ever checking if the primary outcomes and analyses promised in trial registration are actually delivered.

Editors should

  • Require an explicit statement of whether the trial has been registered and where.
  • Insist that reviewers consult trial registration, including modifications, and comment on any deviation.
  • Explicitly label registration dated after patient accrual has started.

spin noCONSORT for abstracts should be disseminated and enforced. A lot of hype and misrepresentation in the media starts with authors’ own spin in the abstract . Editors should insist that main analyses for the preregistered primary outcome be presented in the abstract and highlighted in any interpretation of results.

No more should underpowered in exploratory pilot feasibility studies be passed off as RCTs when they achieve positive results. An orderly sequence of treatment development should occur before conducting what are essentially Phase 3 randomized trials.

Here as elsewhere in reforming psychotherapy research, there is something to be learned from drug trials. A process of intervention development ought to include establishing the feasibility and basic parameters of clinical trials needs to proceed phase 3 randomized trials, but cannot be expected to become phase 3 or to provide effect sizes for the purposes of demonstrating efficacy or comparison to other treatments.

Use of wait list, no treatment, and ill-defined routine care should be discouraged as control groups. For clinical conditions for which there are well-established treatments, head-to-head comparisons should be conducted, as well as including control groups that might elucidate mechanism. A key example of the latter would be structured, supportive therapy that controls for attention and positive expectation. There is little to be gained by further accumulation of studies in which the efficacy of the preferred treatment is assured by comparison to a lamed control group that lacks any conceivable element of affective care.

Evaluations of treatment effects should take into account prior probabilities suggested by the larger literature concerning comparisons between two active, credible treatments. The well-studied treatment of depression literature suggests some parameters: effect size is associated with a treatment are greatly reduced when comparisons are restricted to credible, active treatments; better quality studies; and controls are introduced for investigator allegiance. It is unlikely that initial claims about a breakthrough treatment exceeding the efficacy of existing treatments will be sustained in larger studies conducted by investigators independent of developers and promoters.

Disclosure of conflict of interest should be enforced and nondisclosure identified in correction statements and further penalized. Investigator allegiance should be considered in assessing risk of bias.

Developers of treatments and persons with significant financial gain from a treatment being declared “evidence-supported” should be discouraged from conducting meta-analyses of their own treatments.

Trials should be conducted with sample sizes adequate to detect at least moderate effects. When positive findings from underpowered studies are published,  readers scrutinize the literature for similarly underpowered trials that achieve similarly positive effects.

Meta-analyses of psychotherapy should incorporate p-hacking techniques to evaluate the likelihood that pattern of significant findings exceeds likely probability.

Adverse events and harms should routinely be reported, including lost opportunity costs such as failure to obtain more effective treatment.

We need to shift the culture of doing and reporting psychotherapy research. We need to shift from praising exaggerated claims about treatment and faux evidence generated to  promote opportunities for therapists and their professional organizations.  Instead, it is much more praiseworthy to provide  robust, sustainable, even if more modest claims and to call out hype and hokum in ways that preserve the credibility of psychotherapy.


Click to Enlarge

The alternative is to continue protecting psychotherapy research from stringent criticism and enforcement of standards for conducting and reporting research. We can simply allow the branding of psychotherapies as “evidence supported” to fall into appropriate disrepute.

Category: evidence-supported, mental health care, meta analysis, psychotherapy, Publication bias, research | Tagged , , , | 14 Comments