Troubles in the Branding of Psychotherapies as “Evidence Supported”

Is advertising a psychotherapy as “evidence supported,”  any less vacuous than “Pepsi’s the one”? A lot of us would hope so, having campaigned for rigorous scientific evaluation of psychotherapies in randomized controlled trials (RCTs), just as is routinely done with drugs and medical devices in Evidence-based Medicine (EBM). We have also insisted on valid procedures for generating, integrating, and evaluating evidence and have exposed efforts that fall short. We have been fully expecting that some therapies would emerge as strongly supported by evidence, while others would be found less so, and some even harmful.

Some of us now despair about the value of this labeling or worry that the process of identifying therapies as evidence supported has been subverted into something very different than we envisioned.  Disappointments and embarrassments in the branding of psychotherapies as evidence supported are mounting. A pair of what could be construed as embarrassments will be discussed in this blog.

Websites such as those at American Psychological Association Division 12 Clinical Psychology and SAMHSA’s National Registry of Evidence-based Programs and Practices offer labeling of specific psychotherapies as evidence supported. These websites are careful to indicate that a listing does not constitute an endorsement. For instance, the APA division 12 website declares

This website is for informational and educational purposes. It does not represent the official policy of Division 12 or the American Psychological Association, nor does it render individual professional advice or endorse any particular treatment.

Readers can be forgiven for thinking otherwise, particularly when such websites provide links to commercial sites that unabashedly promote the therapies with commercial products such as books, training videos, and workshops. There is lots of money to be made, and the appearance of an endorsement is coveted. Proponents of particular therapies are quick to send studies claiming positive findings to the committees deciding on listings with the intent of getting them acknowledged on these websites.

But now may be the time to begin some overdue reflection on how the label of evidence supported practice gets applied and whether there is something fundamentally wrong with the criteria.

Now you see it, now, you don’t: “Strong evidence” for the efficacy of acceptance and commitment therapy for psychosis

On September 3, 2012 the APA Division 12 website announced a rating of “strong evidence” for the efficacy of acceptance and commitment therapy for psychosis. I was quite skeptical. I posted links on Facebook and Twitter to a series of blog posts (1, 2, 3) in which I had previously debunked the study claiming to demonstrate that a few sessions of ACT significantly reduced rehospitalization of psychotic patients.

David Klonsky, a friend on FB who maintains the Division 12 treatment website quickly contacted me and indicated that he would reevaluate the listing after reading my blog posts and that he had already contacted the section editor to get her evaluation. Within a day, the labeling was changed to “designation under re-review as of 9/3/12”and it is now (10/16/12) “modest research support.”

David Klonsky is a serious, thoughtful guy with an unenviable job: keeping the Division 12 list of evidence supported treatments updated. This designation is no less important than it once was, but it is increasingly difficult to engage burned out committee members to evaluate the flood of new studies that proponents of particular therapies relentlessly send in. As we will see with this incident, the reports of studies that are considered are not necessarily reliable indicators of the efficacy of particular treatments, even when they come from prestigious, high impact journals.

The initial designation of ACT as having “strong evidence” for psychosis was mainly based on a single, well promoted study, claims for which made it all the way to Time magazine when it was first published.

Bach, P., & Hayes, S.C. (2002). The use of acceptance and commitment therapy to prevent the rehospitalization of psychotic patients: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 70, 1129-1139.

Of course, the designation of strong evidence requires support of two randomized trials, but the second trial was a modest attempt at replication of this study and was explicitly labeled as a pilot study.

The Bach and Hayes  article has been cited 175 times as of 10/21/12 according to ISI Web of Science, mainly  for claims that appear in its abstract: patients receiving up to four sessions of an ACT intervention had “a rate of rehospitalization half that of TAU [treatment as usual] participants over a four-month follow-up [italics added].” This would truly be a powerful intervention, if these claims are true. And my check of the literature suggests that these claims are almost universally accepted. I’ve never seen any skepticism expressed in peer reviewed journals about the extraordinary claim of cutting rehospitalization in half.

Before reading further, you might want to examine the abstract and, even better, read the article for yourself and decide whether you are persuaded. You can even go to my first blog post on this study where I identify safe some of the things to look for in evaluating the claims. If these are your intentions, you might want to stop reading here and resume after considering these materials.

Warning! Here comes the spoiler.

  • It is not clear that rehospitalization was originally set as the primary outcome, and so there is a possible issue of a shifting primary outcome, a common tactic in repackaging a null trial as positive. Many biomedical journals require that investigators publish their protocols with a designated primary outcome before they enter the first patient into a trial. That is a strictly enforced requirement  for later publication of the results of the trial. But that is not yet usually done for RCTs testing psychotherapies.The article is based on a dissertation. I retrieved a copy andI found that  the title of it seemed to suggest that symptoms, not rehospitalization, were the primary outcome: Acceptance and Commitment Therapy in the Treatment of Symptoms of Psychosis.
  • Although 40 patients were assigned to each group, analyses only involved 35 per group. The investigators simply dropped patients from the analyses with negative outcomes that are arguably at least equivalent to rehospitalization in their seriousness: committing suicide or going to jail. Think about it, what should we make of a therapy that prevented rehospitalization but led to jailing and suicides of mental patients? This is not only a departure from intention to treat analyses, but the loss of patients is nonrandom and potentially quite relevant to the evaluation of the trial. Exclusion of these patients have substantial impact on the interpretation of results: the 5 patients missing from the ACT group represented 71% of the reported rehospitalizations  and the 5 patients missing from the TAU group represent 36% of the reported rehospitalizations in that group.
  • Rehospitalization is not a typical primary outcome for a psychotherapy study. But If we suspend judgment for a moment as to whether it was the primary outcome for this study, ignore the lack of intent to treat analyses, and accept 35 patients per group, there is still not a simple, significant difference between groups for rehospitalization. The claim of “half” is based on voodoo statistics.
  • The trial did assess the frequency of psychotic symptoms, an outcome that is closer to what one would rely to compare to this trial with the results of other interventions. Yet oddly, patients receiving the ACT intervention actually reported more, twice the frequency of symptoms compared to patients in TAU. The study also assessed how distressing hallucinations or delusions were to patients, what would be considered a patient oriented outcome, but there were no differences on this variable. One would think that these outcomes would be very important to clinical and policy decision-making and these results are not encouraging.

This study, which has been cited 64 times according to ISI Web of Science, rounded out the pair needed for a designation of strong support:

Gaudiano, B.A., & Herbert, J.D. (2006). Acute treatment of inpatients with psychotic symptoms using acceptance and commitment therapy: Pilot results. Behaviour Research and Therapy, 44, 415-437.

Appropriately framed as a pilot study, this study started with 40 patients and only delivered three sessions of ACT. The comparison condition was enhanced treatment as usual consisting of psychopharmacology, case management, and psychotherapy, as well as milieu therapy. Follow-up data were available for all but 2 patients. But this study is hardly the basis for rounding out a judgment of ACT as efficacious for psychosis.

  • There were assessments with multiple conventional psychotic symptom and functioning measures, as well as ACT specific measures. The only conventional measure to achieve significance was distress related to hallucinations and there were no differences in ACT specific measures. There were no significant differences in rehospitalization.
  • The abstract puts a positive spin on these findings: “At discharge from the hospital, results suggest that short-term advantages in effect of symptoms, overall improvement, social impairment, and distress associated with hallucinations. In addition, more participants in the ACT condition reach clinically significant symptom improvement at discharge. Although four-month rehospitalization rates were lower in the ACT group, these differences did not reach statistical significance.”

The provisional designation of ACT as having strong evidence of efficacy for psychosis could have had important consequences. Clinicians and policymakers could decide that merely providing three sessions of ACT is a sufficient and empirically validated approach to keep chronic mental patients from returning to the hospital and maybe even make discharge decisions based on whether patients had received ACT. But the evidence just isn’t there that ACT prevents rehospitalization, and when the claim is evaluated against what is known about the efficacy of psychotherapy for psychotics, it appears to be an unreasonable claim bordering on the absurd.

The redesignation of ACT as having modest support was based on additional consideration of a follow-up study of the Bach and Hayes, plus an additional feasibility study that involved 27 patients randomized to either to treatment as usual or 10 sessions of ACT plus treatment as usual. Its stated goal was to investigate the feasibility of using ACT to facilitate emotional recovery following psychosis, but as a feasibility study, included a full range of outcomes with the intention of deciding which would be important for assessing the impact of ACT in this population. The scales included the two subscales of the Hospital Anxiety and Depression Scale (HADS), the positive and negative syndrome scale, an ACT specific scale, and a measure of the therapeutic alliance.  Three of the patients assigned just treatment as usual dropped out and so intent to treat analysis were not conducted. With such a small sample, it is not surprising that there were no differences on most measures. The investigators noted that the patients receiving ACT and had fewer crisis contacts over the duration of the trial, but it is not clear whether this is simply due to the treatment as usual group not having regular treatment and therefore having to resort to crisis contacts.

The abstract of the study states “ACT appears to offer promise in reducing negative symptoms, depression and crisis contacts in psychosis”, which is probably a bit premature. Note also that across these three trials, there is a shift in the outcome to which the investigators point as evidence for the efficacy of ACT for psychosis. The assumption seems to be that any positive result can be claimed to represent a replication, even if other variables were cited for this purpose among the other studies.

Overall, this trial would also be rated as having high risk of bias because of the lack of intent to treat analyses and the failure to specify a primary outcome among the battery that was administered, but more importantly, it would simply be excluded from meta-analyses with which I have been associated because of too few patients in it. A high risk of bias plus too few patients discourages any confidence in these results.

Is treating PTSD with acupoint stimulation supported by evidence ?

Whether or not ACT is more efficacious than other therapies, as its proponents sometimes claim, or whether it is efficacious for psychosis, is debatable, but probably no one would consider ACT anything other than a bona fide therapy. The same does not hold for Emotional Freedom Therapy (EFT) and its key component, acupoint.  I’m sure there was much consternation at APA and Division 12 when stories circulated on the Internet that APA had declared EFT to be evidence supported.

Wikipedia offers the following definition of EFT:

Emotional Freedom Techniques (EFT) is a form of counseling intervention that draws on various theories of alternative medicine including acupuncture, neuro-linguistic programming, energy medicine, and Thought Field Therapy. During an EFT session, the client will focus on a specific issue while tapping on so-called “end points of the body’s energy meridians.”

Writing in The Skeptical Inquirer, Brandon Gaudiano and James Herbert argued that there is no plausible mechanism to explain how the specifics of EFT could add to its effectiveness and they have been described as unfalsifiable and therefore pseudoscientific. EFT is widely dismissed by skeptics, along with its predecessor, Thought Field Therapy and has been described in the mainstream press as “probably nonsense.”[2] Evidence has not been found for the existence of acupuncture points, meridians or other concepts involved in traditional Chinese medicine.

The scathing Gaudiano and Herbert critique is worth a read and calls attention to claims of EFT by proxy: patients improve when therapists tap themselves rather than the patients! My imagination runs wild: how about televised sessions in which therapists tap themselves and liberate thousands of patients around the world from their PTSD?

According to David Feinstein, aproponent of EFT, in including a chapter on Thought Field Therapy in an anthology of innovative psychotherapies, Corsini (2001) acknowledged that it was “either one of the greatest advances in psychotherapy or it is a hoax.”

Claims have been made for acupoint that even proponents of EFT consider “provocative,” “extraordinary,”  and “too good to be true.” An article published in Journal of Clinical Psychology (not an APA journal), reported that 105 people were treated in Kosovo for severe emotional reactions to past torture, rape, and witnessing loved ones being burned or raped. Strong improvement was observed in 103 of these patients, despite an average of only three sessions. For comparison purposes, exposure therapy involves at least 15 sessions in the literature claims nowhere near this efficacy. However, even more extraordinary results were claimed for the combined sample of 337 patients treated in visits to Kosovo, Rwanda, the Congo, and South Africa. The 337 individuals expressed 1016 traumatic memories of which 1013 were successfully resolved, resulting in substantial improvement in 334 patients. Unfortunately the details of this study remain on unpublished, but claims of these results appear in a forthcoming article in the APA journal Review of General Psychology.

Reports circulating on the Internet that APA had declared EFT to be an evidence supported approach stemmed from a press release by the EFT Universe that cited a statement from the same Review of General Psychology article:

A literature search identified 50 peer-reviewed papers that report or investigate clinical outcomes following the tapping of acupuncture points to address psychological issues. The 17 randomized controlled trials in this sample were critically evaluated for design quality, leading to the conclusion that they consistently demonstrated strong effect sizes and other positive statistical results that far exceed chance after relatively few treatment sessions. Criteria for evidence-based treatments proposed by Division 12 of the American Psychological Association were also applied and found to be met for a number of conditions, including PTSD (Feinstein, 2012).

Feinstein had been developing his claims about energy therapies such as EFT meeting the Division 12 criteria for a while. In a 2008 article in the APA journal Psychotherapy Theory, Research, Practice, Training, he declared

although the evidence is still preliminary, energy psychology has reached the minimum threshold for being designated has an evidence-based treatment, with one form having met the APA division 12 criteria as a “probably efficacious” treatment for specific phobias; another for maintaining weight loss.

In this 2008 article, Feinstein also cited a review in the online book review journal of APA in which Ilene Selrin, Past President of APA’s Division of Humanistic Psychology praised Feinstein’s book for its “valuable expansion of the traditional biopsychosocial model of psychology to include the dimension of energy” and energy psychology as representing “a new discipline that has been receiving attention due to its speed and effectiveness with difficult cases.”

The reports that EFT had been designated as an evidence supported treatment made the rounds for a few months, sometimes with the clarification that EFT met the criteria, but had not yet been labeled as evidence supported by Division 12. In some communities, stories about EFT or –as it was called– tapping therapy made the local TV news. KABC news Los Angeles titled a story,”‘Tapping’ therapy can relieve anxiety, stress, researchers say” and got an APA spokesperson to provide a muted comment

 ”Has this tapping therapy been proven effective? We don’t think so at this point,” said Rhea Farberman, Executive Director for Public and Member Communications at the APA.

The comment went on to say that APA viewed stress and anxiety as serious but treatable issues for some persons and cognitive behavior therapy recommended, but not tapping therapy.

What do these incidents say about branding of psychotherapies as evidence supported?

I will explore this issue in greater depth in a future blog post, but for now we are left with some questions.

The first incident involved designation of a psychotherapy as having strong evidence of efficacy for psychosis, but was quickly changed first to under review and then to modest support. The precipitant for this downgrading seems to be blog posts that revealed the abstract of the key study to be misleading. Designation of a therapy as having strong evidence for its efficacy requires two positive randomized controlled trials. The second trial was described as a pilot study explicitly aimed at replicating the first one. Like the first one, its abstract declared positive findings. However, this study failed to replicate the first study’s claimed reduction in hospitalization, and a cursory examination of the results section revealed that this study, like the study that it attempted to replicate, was basically a null trial.

  • Do the current criteria employed by Division 12-only 2 positive trials and no attention to size or quality- set too low a bar for a therapy receiving the seemingly important branding of having strong evidence?
  • The revised status of ACT for psychosis is that it has modest support. But how does two null trials published with confirmatory bias constitute modest support?
  • Are there pitfalls in uncritically accepting claims in the abstracts of articles appearing in prestigious journals like JCCP?
  • More generally, to what extent do the shortcomings of articles appearing in prestigious journals like JCCP warrant skepticism, not only by reviewers for Division 12, but consumers more generally?
  • Should we expect a prestigious journals like JCCP to encourage and make a place for post publication peer review of the articles that have appeared there?
  • Should revised criteria for evidence supported therapies not just count whether there are two or only one positive trial, but incorporate formal quality ratings of trials for overall quality and risk of bias?

The second incident involves rumors of APA having designated as evidence supported a bizarre therapy with extravagant claims of efficacy. The rumor was based on a forthcoming review in an APA Journal that indicated that EFT had sufficient number of positive randomized trials to meet APA division 12 criteria for evidence supported. It was left to a media person from APA to clarify that APA did not endorse this therapy, but it was unclear on what basis this declaration was made.

  • If ACT for psychosis has modest support, where does EFT stand when evaluated by the same criteria?
  • Can sources other than APA Division 12 apply the criteria to psychotherapies and declare the therapies as warranting evidence-based status? If not, why not?
  • Do consumers, as well as proponents of innovative and even strange therapies, deserve evaluation with formal criteria by APA Division 12 and designation of the therapies not only as warranting a designation of “strong evidence” if they meet these criteria, but alternatively as having demonstrated a failure to accumulate evidence of efficacy, and even as having demonstrated possible harm?
  • If APA Division 12 takes on the task of publicizing the evidence based status of psychotherapies, does it thereby assume a responsibility to alert policy makers and consumers of therapies that fail to meet these criteria?
  • If application of the existing Division 12 criteria warrants EFT as having strong evidence of efficacy, what does that say about the adequacy of these criteria?

To be continued……

Creative Commons License
This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution 3.0 Unported License.

This entry was posted in Uncategorized. Bookmark the permalink.

19 Responses to Troubles in the Branding of Psychotherapies as “Evidence Supported”

  1. James Herbert says:

    Some of Jim’s concerns raised here about ACT are well taken, but from other things he’s written on the subject one gets the impression that he has an ax to grind with that particular approach. It’s worth noting that the psychosis data were re-analyzed based on his helpful critiques and suggestions, and those results are now published; see: Back, Gaudiano, Hayes, & Herbert; Acceptance and commitment therapy for psychosis: intent to treat, hospitalization outcome and mediation by believability; Psychosis: Psychological, Social and Integrative Approaches (http://www.tandfonline.com/doi/full/10.1080/17522439.2012.671349).

    But his bigger concern here is not about ACT or even EFT, but about the EST project, and on that I think he’s right on point. His concerns echo back to those that we (and many others) have raised over the years about the EST project. The program to identify EST’s was very well intentioned, but was doomed from the beginning by fatal flaws in the criteria. For those interested, see:

    Herbert, J. D. (2003). The science and practice of empirically supported treatments. Behavior Modification, 27, 412-430.

    Herbert, J. D., & Gaudiano, B. A. (2005). Moving from empirically supported treatment lists to practice guidelines in psychotherapy: The role of the placebo concept. Journal of Clinical Psychology, 61, 893-908.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • James Coyne PhD says:

      Thanks, James Herbert, for the references that I I am sure I will find useful for writing future blogs on these topics, just as I found your paper with Bruno on Emotional Freedom Technique in the Skeptical Inquirer useful this round. I agree with you, the point of my blog was about the limitations of the project to identify evidence supported treatments that now is falling prey to flaws that were built in at the outset. As I will say in future blogs, I think that we need to stop relying on committees that are weakened by politics and unspoken conflicts of interest to decide what is evidence supported. Instead, we should construct in a lively and contentious debate a set of standards by which we can decide for ourselves what is evidence supported, what is inert, and what is harmful.
      I appreciate the ACT community’s responsiveness to my past criticisms, but when I examined the re-analyses that you cite, they turned out to be based on combining results from two different trials in an unconventional fashion. There is a high risk of bias in such a post hoc readjustment of sample size and reanalysis.
      I am a closet Bayesian and all has gone on after Steve Hayes’ publicity blitz about having halved rehospitalization rates of mental patients with three sessions of ACT has not moved my assessment that such results are highly improbable and need to be treated with skepticism.

      Here is a link to the article that you mentioned that is not behind a pay wall.

      http://drexel.edu/~/media/Files/psychology/labs/formanherbert/publications/Bach%20et%20al%202012%20psychosis%20mediation.ashx
      .

      VN:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
  2. Neuroskeptic says:

    Excellent post!

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  3. Bryan Knight says:

    Wow. Talk about “how many angels can dance on the head of a pin.”

    You are right that real “evidence-based” therapy would be invaluable. However, as I wrote in (a href=”http://www.hypnosisdepot.com/therapy-delusion.php” >The Therapy Delusion the first step in seeking efficacious therapies would be to lose our belief that metaphors describe reality.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  4. Steven C. Hayes says:

    I appreciate Jim’s skepticism … I share it. These results DO seem implausible and I’m in that Bayesian camp too.

    We saw a pretty similar pattern in the replication by Gaudiano and Herbert tho
    (that is not enough to bump me personally into the “I believe” camp … when I
    describe the studies in talks I still say I’m not sure I believe it).

    As Jim mentions in his response we went back as a result of his criticisms and did an intent to treat analysis and a longer term follow up (and another mediational analysis).

    The Gaudiano and Herbert study used the same measures and protocol — it seemed reasonable to examine them in a combined analysis. I need to be schooled on what was unconventional or inappropriate (in your response to James) about the data analysis. In fact, a couple of the analyses we did there were just the ones you’d called for in an earlier blog post. I’m open to ideas.

    The fact is we just need more research. What we need is a large funded trial.

    I personally have just failed at it. I do know how to write grants but I think I have written like 5 of them in this area and got no where. I mean zip.
    At my age I’m not sure I can keep banging my head on the wall … but fortunately ACT is being developed in an open and non-proprietary way, so other folks out there are picking up the ball on their own.

    There is a group of UK researchers who finally have funding to do a larger trial so we will know in a few years whether this ACT opening is a real lead or a bunch of nonsense.

    I would also like to say publicly that I personally agree with the redesignation to “modest support” (before the change I communicated with David on that and expressed my support for it). Seems fairer.

    As far as the “publicity blitz” that is just what reporters do, Jim. Its fairer to judge scientists by what they write … the media is a creature of its own. We’ve been cautious in our science writings (haven’t we? I’m asking seriously — if there is an intemperate claim out there I would like to track it down).

    For readers interested here are a few more data-based articles on ACT for psychosis

    Here is the one year follow up of the Bach and Hayes study: Bach, P., Hayes, S. C. & Gallop, R. (2012). Long term effects of brief Acceptance and Commitment Therapy for psychosis. Behavior Modification, 36, 167-183. It was done as our first response to Jim’s criticisms actually.

    There was also mediational analyses of the Gaudiano and Herbert dataset: Gaudiano, B. A., Herbert, J. D., & Hayes, S. C. (2010). Is it the symptom or the relation to it? Investigating potential mediators of change in Acceptance and Commitment Therapy for psychosis. Behavior Therapy, 41, 543-554.

    White, R.G., Gumley, A.I., McTaggart, J., Rattrie, L., McConville, D., Cleare, S, & Mitchell G. (2011). A feasibility study of Acceptance and Commitment Therapy for emotional dysfunction following psychosis. Behaviour Research and Therapy, 49, 901-907.

    Anyway, I personally value science skeptics. Jim has helped us with his criticisms and I now send him new studies routinely.

    In the long run this is not about brands, or personalities — its about processes and procedures that can be useful to others. That is what the science game is really about. We will all be forgotten soon enough … but knowledge lives on. Best to keep the focus there. The rest is not important

    - S

    Steven C. Hayes
    Foundation Professor
    University of Nevada

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • James Coyne PhD says:

      Steve, thanks for joining in the discussion and for contributing to a climate in which skepticism is encouraged. At this point, we just don’t know whether ACT has the effect claimed for rehospitalization ands we agree that we start with some skepticism about whether any therapy can have this effect. I also think however is best not to stake the claim for efficacy of a psychotherapy on this particular variable.

      Regardless, my issue with the combined analyses is that the decision to combine them is post hoc and I think you would have a hard time finding precedents in the existing literature. Such post hoc decisions increase the likelihood of bias. Much preferred is to set stopping rules/number of patients ahead of time, prior to finding out what the results are, and sticking to that.

      I look forward to seeing the results of future trials. I see If you go to http://www.clinicaltrials.gov and search for closed trials, which include completed trials, as well as trials that folded without finishing, for example, and use the term “Acceptance and Commitment” you get a list of 13 trials (link below). The first two are for psychosis and delusions, respectively. I look forward to seeing the resulting publications

      Best

      VN:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
  5. Keith Laws says:

    A couple of quick things – everything written here is a replay of what we have seen in the context of CBT for psychosis

    A few studies showing great promise, followed by meta analyses (including Cochrane) suggesting same, implementation by govt organisations (in UK NICE).
    Some remark on poor methods (like us: Lynch Laws & McKenna 2010) and everyone thinks we are party poopers – accusing us, as above, of having an axe to grind.
    And then everyone notices just how poorly conducted are many of the trials: no blinding, no active control group, using rehospitalisation as a proxy for relapse etc. More meta analyses then showing no great effect or nothing greater than far cheaper atheroetical alternatives (e.g. befriending, supportive counselling) especially in blind studies – all compunded by publciation bias…

    Then the defence arguments are made that its a power issue (we need more studies with greater samples ad infinitum)

    While little has been done in ACT, much has in CBT (in UK many large multi-million pound funded trials) and still we get studies with no controls (or comparing to TAU at best) or no blinding – under the guise of ‘pilot’. I will no doubt come back in a few years and see that the same is occuring here…
    skeptically yours

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  6. Jazi zilber says:

    hayes,

    The problem with combining two trials is that there is amwidening space of possibilities, which requires a corrective bonferroni multiplier.

    You can or not combine two studies, and who knows which other studies you could have combined, etc.

    So even when accepting after the fact decisions on how to sum up things, you still got some (unclear) space of possibilities, for whom you need to multiply..

    Is the p-value of the results < 0.001? If the outcome is strong enough it stays interesting.

    Generally, the technical solution (never do ad hoc calculation, no matter the justification) has merits, as it avoids the pitfalls. It is clealry better to design systems so that little is done ex-post.

    Great post, James

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  7. James C. Coyne says:

    Discussion of my recent blog post has spilled over onto the list serves. Here are some excerpts from the Friends of Positive Psychology listserv. I am not allowed on that listserv. I briefly signed on and was a lurker and then when Marty Seligman posted statements about attitude influencing survival in cancer, I offered a PDF of a review of evidence to the contrary. It was my first and last post and I was taken off the list serve with just a brief email from a masters student in the positive psychology program. No worries, somebody somehow rigged a way for me to get daily summaries. Here’s a sampling of what is being said there this morning:

    A lesson from James Herbert, Steve Hayes, and the Acceptance and Commitment Therapy on how to respond to criticism:
    http://blogs.plos.org/mindthebrain/2012/10/22/troubles-in-the-branding-of=
    -psychotherapies-as-evidence-supported/

    See the great comment exchange at the bottom. Instead of ruling out or tossing ad hominems at James Coyne or any other skeptic, they focus on the ideas. And as many on this list know, he isn’t the friendliest, open-minded, cuddliest person on the block but he’s one smart bloke. And his criticisms often have great merit….even if they bring up uncomfortable emotions and aren’t softened by a field padded with roses and puppies.

    The archives over the past few years offer insight into similarities and differences in this community.

    AND
    Build skepticism into the foundation; if done in the inside, then the work will be battle tested for the outside. In general, this community is still uber-defensive despite being around for 14 years or if according to Richie Davidson 20 years….its time to show a more mature confident disciplinethat doesn’t have to be rattled by media, journal articles, books, and online comments on this listserv. Roll with resistance (see motivational interviewing).

    if you are in a leadership position in IPPA or elsewhere, pass this around.Would love to see some responses. Marty, thoughts?
    AND

    Great example.

    Btw, skepticism is a tough and very unpopular undertaking. Being
    positive is much more socially accepted and considered “progress”.

    Yet, where progress goes via multiple trials and errors, fast and
    decisive negation of false starts is very very important (Taleb,
    forthcoming)

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  8. Stropy says:

    I do not even know how I stopped up here, but I thought this publish used to be good. I don’t understand who you’re however certainly you’re going to a well-known blogger should you aren’t already. Cheers!

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  9. Pingback: “”Troubles in the Branding of Psychotherapies as “Evidence Supported” “ | notessimple

  10. BISE says:

    I run my own blogand I came here by mistake. I read your articles and I found that they’re fascinating! I want to use them on my blog. I want do that without your permission, so say OK. I’m greeting warmly.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  11. Mellisa says:

    I love reading through a post that can make people think.

    Also, thanks for permitting me to comment!

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  12. org.br says:

    Very energetic post, I enjoyed that a lot. Will there be a
    part 2?

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  13. Pingback: [Reblog] Troubles in the Branding of Psychotherapies as “Evidence Supported” « Health and Medical News and Resources

  14. Pingback: What We Need to Do to Redeem Psychotherapy Research | Quick Thoughts

Add Comment Register



Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>