How Understanding Psychosis could have been more credible and trustworthy

British psychological society

Click to enlarge

As promised, this issue of Mind the Brain explains how the hanging out the truthBritish Psychological Society Division of Clinical Psychology ’s Understanding Psychosis could have been much more credible and trustworthy.

I point to well-founded skepticism about like-minded, self-selected groups representing single professions lacking any cultural diversity trying to tell clinicians and policymakers about how health services ought be re-organized. The folly of Division of Clinical Psychology’s way of doing things is compounded by excluding key consumer stakeholders. I will provide some standards and procedures that were blatantly ignored in the writing and dissemination of their recommendations.

The Division of Clinical Psychology is preparing a companion document about depression. I hope there is time for their adopting international standards. But they would have to open themselves to diversity and desegregate, allowing ethnic minorities, especially African and British Blacks a seat at the table. I will explain why their systematic exclusion of this group from the deliberations is particularly egregious, given gross inequalities in the services they receive, often from almost uniformly white clinical psychologists.

I take seriously the authors claim that they wanted to be provide an authoritative source of information for to mental health service users, their family members, and other professionals and policymakers and members of the community attempting to decide what the best policies for dealing with persons described as suffering from psychosis or schizophrenia. But I don’t accept the document simply because the authors claim they are experts or that they are creating a paradigm shift.

Skepticism should be raised when professional groups crow too loudly about their expertise and creating a shift in paradigm. Rhetorically, professionals fare better when they show what they have to offer and leave for others to decide whether they should be labeled “experts” or that they are causing a paradigm shift.

paradigm-shift-cartoon

Click to enlarge

I recall Dan Haller, former editor of Journal of Clinical Oncology poking fun at authors submitting manuscripts that they claimed represented paradigm shifts. Maybe in hindsight, Galileo, and, over Einstein deserve that label, but no paper he had ever reviewed about chemotherapy, radiation treatment, for immunotherapy earned it. Haller felt authors’ claim of making a paradigm shift simply embarrassed themselves.

When I subjected the 180 page document to my usual skeptical, critical scrutiny, its credibility and trustworthiness simply didn’t hold up. It seemed to be a collection of carefully selected and edited quotes and minimal, but unsystematic reference to the literature. It seemed to crassly sacrifice the well-being of persons with psychosis and schizophrenia – white, African and British black, and other groups – to professional self-interests of a small group of psychologists.

To an American like me, Understanding Psychosis seems like a bit of old-fashionedBengal-governor-during-British-rule colonial administratorBritish colonial administration. Clad in pith helmets, the British clinical psychologists went out and recruited a few supporters who shared their views and suppressed silenced the rest of service users and their families – pretending they don’t even exist – who would be so affected by their proposals. And as I noted in my last blog post, there are grounds to doubt that a good proportion of the supporters whom they quote are even service users.

When I raised the issues of consensus and process in Understanding Psychosis on Twitter in November 2014, I got an immediate response from the official Division of Clinical Psychology Twitter account

Click to enlarge

Click to enlarge

To which I replied

Please click to enlarge

Please click to enlarge

My “debunking” – if that’s what@DCP wants to call it – involves systematic gathering relevant evidence and evaluating it by transparent standards that others have agreed upon. When I formally do this in peer-reviewed articles, I typically involve other people as a check on my biases, as well as procedures by which readers can decide for themselves on the validity of my conclusions. When I am flying solo, that needs to be taken into account, and readers should start with greater skepticism. I am more dependent on sufficiently documenting my evidence in order to persuade them.

Some questions are clearly defined enough to proceed with a systematic search for relevant evidence: “the screening for psychological distress improves patient outcomes?”

But many questions like “do we abandon psychiatric diagnosis” or “how do we best organize services to ensure better patient outcomes?” involve potentially controversial decisions about how to sharpen the questions in order to gather relevant evidence. It’s best to get a diversity of opinions of both professionals and service consumers to define the range of possibilities. There need to be some checks on biases, with the hope that these can be overcome by some consensus process among people starting with clear differences of opinion. That is not just an ideal, that’s a necessity if some professional group is going to claim authority for its recommendations. I am typically not operating in that context, and so the credibility of what I in my co-authors come up with the strength of evidence, and we leave for others decisions about how or whether  recommendations will be implemented.

There are some widely accepted standards for bringing relevant stakeholders together, reviewing available evidence, and formulating recommendations. There is lots of evidence about the consequences when these procedures are followed.

But before getting into them, let me describe how I came to be appreciative of both the necessity for the standards for professional organizations formally making policy recommendations and the existence of rules by which they should proceed and be evaluated.

Our 2008 JAMA systematic review and meta-analysis of screening for depression in cardiac patients and reactions from the American Psychiatric Association.

Our paper was

Thombs, B. D., de Jonge, P., Coyne, J. C., Whooley, M. A., Frasure-Smith, N., Mitchell, A. J., … & Ziegelstein, R. C. (2008). Depression screening and patient outcomes in cardiovascular care: a systematic review. JAMA, 300(18), 2161-2171.

Our international group of authors had published key papers and chapters in a book onscreening for depression the topic of screening for depression, as well as the role of depression in cardiovascular disease (CVD). We neither proclaimed ourselves “experts” nor had the endorsement of a professional organization backing up our conclusions. But we identified and followed well defined standards for turning clinical and policy issues into topics for systematic review and meta-analysis. And we were quite transparent in what we did and how it conformed to international standards.

Our conclusion was

The high prevalence of depression in patients with CVD, the adverse health care outcomes associated with depression, and the availability of easy-to-use case-finding instruments make it tempting to endorse widespread depression screening in cardiovascular care. However, the adaptation of depression screening in cardiovascular care settings would likely be unduly resource intensive and would not be likely to benefit patients in the absence of significant changes in current models of care.

The JAMA editors liked the paper enough to invite some of the authors to participate in a live webinair with participants able to telephone and email questions.. The editors of BMJ nominated the paper is one of the eight top papers of the year to be considered in a competition for the top paper.

I was caught off guard when just a few weeks later a paper appeared on the Internet labeled as a American Heart Association Science Advisory with a list of impressive committees signing on to its conclusions and the American Psychiatric Association prominently listed as endorsing the advisory.

The recommendations directly contradicted ours:

Although there is currently no direct evidence that screening for depression leads to improved outcomes in cardiovascular populations, depression has been linked to increased morbidity and mortality, poorer risk factor modification, lower rates of cardiac rehabilitation, and reduced quality of life.  Therefore, it is important to assess depression in cardiac patients with the goal of targeting those most in need of treatment and support services.

And

In summary, the high prevalence of depression in patients with CHD supports a strategy of increased awareness and screening for depression in patients with CHD.

Politics versus rules of making evidence-based decisions

Our conclusions were based on best evidence and transparent rules for evaluating that evidence. The AHA Science Advisory was based on a consensus of professionals – psychologists and psychiatrists – who had vested interests in promoting screening because it would increase their professional opportunities in cardiology settings.

Although publicity for our article had some momentum, the promoters of the AHA Science Advisory jumped into the media with a lot of political power to counter our conclusions, while usually failing to acknowledge who we were and where we had published. The American Psychiatric Association actually assigned a pediatric psychiatrist to become a media contact for their point of view.

I had naïvely thought that best evidence would trump consensus of professionals with obvious self-interests at stake. The weight of evidence was clearly on our side. But one of our cardiologist co-authors, Roy Zigelstein was not at all surprised by the carefully orchestrated reaction.

Roy negotiated us an opportunity with American Heart Journal and Journal of the American Academy of Cardiology to explain the differences between us and the AHA Science Advisory. Although we were up against strong vested interests, cardiologists themselves were not necessarily in agreement with the science advisory.  Actually, the American Heart Association continually updates its evaluations of factors correlated with cardiovascular outcomes as causal factors. To this day, it still has not accepted depression as a causal factor, only a risk marker. The implication is that making changes in depression may not necessarily affect cardiac outcomes.

In our commentary at American Heart Journal, we noted the discrepancy between the results of our meta-analysis and systematic review versus the conclusions of the AHA Science Advisory. We also noted that we were not alone in expressing concern about guidelines issued by the American Heart Association increasingly being based on simple professional consensus and not a systematic review of the evidence. Consequently many of them were not “best evidence.”

“In guidelines we cannot trust”

Our skirmishing with the AHA Science Advisory and American Psychiatric Association occurred at a time when recognition was already growing that the recommendations of professional organizations were untrustworthy. There was documentation of numerous instances in which they were often not evidence-based, but served their self-interests, often at the expense of patient outcomes. Many of the recommendations were for billable procedures from the professional groups who created them that were unnecessary and even harmful to patients.

The title of a later article captured the rampant skepticism of the time:

Shaneyfelt T. In guidelines we cannot trust. Arch Intern Med 2012;172:1633-1634.

There were lots of proposals for reform, like a series that included

Fretheim A, Schunemann HJ, Oxman AD. Improving the use of research evidence in guideline development: 5. Group processes. Health Res Policy Syst 2006;4:17.

guidelines we can trustBut discontent gut all the way to the U.S. Congress, which authorized that the Institute of Medicine (IOM) be given the resources to organize a panel with wide representation to come up with, as the final 250 page document was titled, Clinical Guidelines We Can Trust. You can download a free PDF here.

The rationale for specific procedures spelled out, but in terms of the final product:

To be trustworthy,  guidelines should

  • Be based on a systematic review of the existing evidence.
  • Be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups.
  • Consider important patient subgroups and patient preferences, as appropriate.
  • Be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest.
  • Provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of the recommendations.
  • Be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.

understand coverThe standards seem eminently reasonable in the deliberations by which they were reached is carefully documented. Yet Understanding Psychosis fails miserably as a set of credible policy recommendations by not meeting any of them. It’s because the process of writing the document was so flawed:

  • The British Psychological Society Division of Clinical Psychology professionals did not engage other professionals with complementary viewpoints and expertise.
  • Key stakeholders were simply excluded – primary care physicians, social workers, psychiatrists, , police and corrections personnel who must make decisions about how to deal with disturbed behavior,  and –most importantly- the family members of persons with severe disturbance.
  • There was no clear explicit process to minimize bias and distortion and no transparency as to how the group arrived at particular conclusions.
  • There was no check on the psychologists simply slanting the document to conform to their own narrow professional self-interests.
  • Recommendations were presented without clear grading of the quality of available evidence or strength of recommendations.
  • While there was a carefully orchestrated “show and tell” rollout, it did not involve any opportunities for feedback and modification of recommendations.

In one of a number of passages plagiarized from an earlier paper, Peter Kinderman recently told clinicians from other disciplines to adopt the recommendations of Understanding Psychosis

To return, then, to the issue of communication between professionals; for clinicians, working in multidisciplinary teams, the most useful approach would be to develop individual formulations; consisting of a summary of an individual’s problems and circumstances, hypothesis about their origins and possible therapeutic solutions. As with direct clinical work, such an approach would yield all the benefits of the traditional ‘diagnosis, treatment’ approach without its many inadequacies and dangers. This would require all clinicians— doctors, nurses and other professionals—to adopt new ways of thinking.

Why should these professionals do the bidding of a small group of self-serving psychologists? They were not involved in the process of constructing these recommendations and the psychologists failed to provide appropriate evidence. There is no evidence that this would improve patient outcomes.

A special pleading for marginalized and silenced Black African clients who were getting poor care.

Enter “African” as a search term in the 180 page Understanding Psychosis and you come up with only a brief mention on page 46 that fails to acknowledge the poor outcomes that tradition African Black are disproportionately achieving in outpatient care. Even if there are few, if any black members of the BPS Diision of Psychology and even if there were no blacks involved in the writing of Understanding Psychosis, surely there was some awareness of the gross disparities in outcomes that are achieved in outpatient care for psychosis and schizophrenia. A recent paper added further evidence to what was already known:

  • Early Intervention Services (EIS) have little effect on the much higher admission and retention rates of Black African clients.
  • There are low rates of GP involvement and high rates of police detention.
  • Poor outcomes were most marked in Black African women (7-8x  greater odds than White British women).
  • A post-hoc analysis showed that pathways to care and help-seeking behavior partially explained these differences.

Overall

In an increasingly outcome-driven and evidence-based era, EIS need to demonstrate a significant positive impact on detecting and treating psychosis early, across all groups. Our findings, when compared with UK studies from the pre-EIS era [5], suggest no improvement in the inequality between Black African patients with FEP and White British patients in terms of experiences of admission and detention. The high rates of detention and hospital admission overall are likely to have substantial implications for continuing engagement. The rate of detention is particularly elevated in Black African patients at 60% (Table 2). A disconcerting finding is of even higher rates in certain groups than prior to introduction of EIS, especially in women. While there is overall evidence that the EIS model is a cost-effective [31] means of engaging hard-to-reach young people, it would seem not all groups are being reached in ways that minimise stigma and trauma. Of note, a recent systematic review of initiatives to shorten DUP [32] concluded that establishing dedicated services for people with FEP does not in itself reduce DUP. This is despite evidence that longer DUP is associated with poorer outcomes [33],[34].

“In an increasingly outcome-driven and evidence-based era,” the British Psychological Society Division of Clinical Psychology had better involve a broader and more ethnically diverse range of opinions and more careful consideration of available evidence if they are going to be taking seriously.

Counterpoint from Richard Pemberton, UK Chair of the British Psychological Society Division of Clinical Psychology:

Your approach to debate and tendency to personalise professional differences however means that many senior people don’t take you seriously and/or aren’t willing to get in the same room as you. Describing the very senior and prestigious group of researchers who were co-authors of our recent psychosis publication as either ‘stoned or drunk’ is a case in point? Doing this in private would be testing but putting this out into the public domain certainly breaches UK professional ethical codes. I am sure that you sincerely believe that the report is deeply flawed and highly problematic but I doubt that you actually believe that we are all sitting around under the influence of drugs and alcohol producing 180 page publications.

Category: antipsychotics, evidence-supported, professional organizations, psychosis, psychotherapy, schizophrenia | Tagged , , , , | Leave a comment

“Understanding Psychosis and Schizophrenia” and mental health service users

understand coverDoes Understanding Psychosis and Schizophrenia exploit, disrespect, and marginalize service users?

Genre confusion.

The 180-page Understanding Psychosis and Schizophrenia produced by the British Psychological Society Division of Clinical Psychology is a puzzling document. We need to know its genre to decide what standards we apply in evaluating it. The authors tell us:

The report is intended as a resource for people who work in mental health services, people who use them and their friends and relatives, to help ensure that their conversations are as well informed and as useful as possible. It also contains vital information for those responsible for commissioning and designing both services and professional training, as well as for journalists and policy-makers. We hope that it will help to change the way that we as a society think about not only psychosis but also the other kinds of distress that are sometimes called mental illness.

“Well-informed” by what or whom? How is the information “vital”? Does “vital” assume “trustworthy” and “credible”?

As I will cover in a later blog issue, the document strikingly lacks the transparency that it would need to be taken seriously.  Understanding Psychosis conforms to none of the well-defined processes and standards – checks and balances – expected to be met by professional organizations producing a report aimed at policy-makers and the general public.

mental ellf1For now, note these psychologists did not engage other professionals with complementary viewpoints and expertise. And the writing  was closed to anyone not already expressing strongly held particular opinions. When critics nonetheless provided a detailed analysis of some crucial points at the popular blog, Mental Elf, the authors of Understanding Psychosis retweeted and favorited a denunciation of them as a “circle jerk,” i.e., mutually masturbating.

circle jerk

Please click to enlarge

how vulgar

Key stakeholders were simply excluded – primary care physicians, social workers, psychiatrists, police and corrections personnel who must make decisions about how to deal with disturbed behavior, and –most importantly- the family members of persons with severe disturbance. There was no check on the psychologists simply slanting the document to conform to their own narrow professional self-interests, which we are asked to accept as “expertise.”

Is Understanding Psychosis evidence-based?

Understanding Psychosis occasionally cites some empirical findings, but can’t be seen as evidenced-based. That would require transparent, systematic strategies for gathering, interpreting, and integrating evidence that are simply not there.

Indeed, I think it is an excellent document for PhD students and trainees to practice debunking the creation of false authority by selective citation and miscitation and ignoring of contradictory studies. I suggest that they arm themselves with Google Scholar and tools provided in

Greenberg, S. A. (2009). How citation distortions create unfounded authority: analysis of a citation network. BMJ, 339.

Then start checking the citations provided for seemingly evidence-based statements in Understanding Psychosis. Ask questions like “What relevant studies are not cited? What studies are misinterpreted or simply cited for findings they did not contain?” Go to Google Scholar or Web of Science and find out.

For instance, take the opinion

In view of the problems with diagnoses, many researchers and clinicians are moving away from using them, and recent high-profile reports have recommended this. 55 56.

Check the references and see that the authors of Understanding Psychosis are the “many researchers and clinicians.” They are praising their own opinion pieces as “high-profile.”

55. British Psychological Society (2013). Division of Clinical Psychology position statement on the classification of behaviour and experience in relation to functional psychiatric diagnoses: time for a paradigm shift. Leicester: British Psychological Society.

56. Division of Clinical Psychology (2011). Good practice guidelines on the use of psychological formulation. Leicester: British Psychological Society.

The authors of Understanding Psychosis would have embarrassed themselves if they stated outright “It is our opinion that…and we consider our opinion high-profile and you should be duly impressed.” They depend on readers not checking references.

Argument from cherry picked quotes.cherrypicking

Understanding Psychosis is a collection of quotes. We might be inclined to interpret this as a strength, a sign of collaborative  participatory research.

Or maybe this represents qualitative research allowing  people to speak for themselves, rather than requiring that their experiences be processed through others’ filters and concepts.  But bona fide, credible qualitative research requires that biases of  investigators not intrude upon what they report.  Some controls must be visibly present preventing the investigators from doing so.

Quotes are carefully selected to support by the psychologists opinions expressed before the document was prepared – like 15 years ago in their Recent Advances in Understanding Mental Illness and Psychotic Experiences.

Many quotes are not from people suffering from schizophrenia. In most instances, we are not given sufficient information to determine this.  The authors systematically withhold information that would allow readers to determine who is and who is not a service user.

In this issue of Mind the Brain, I examine implications of this heavy dependence on these particular quotes. I will question whether Understanding Psychosis involves using and even exploiting service users, pitting more highly functioning ones against those who are functioning less well and their families who have to deal with them when they cannot take care of themselves.

Where do the quotes in Understanding Psychosis come from?

Some quotes were simply pasted in from the 2000 Recent Advances in Understanding Mental Illness and Psychotic Experiences.

Presumably people had relevant experience in the interim for our grasping the relevance to what it like living with schizophrenia and other psychoses – if that was actually their circumstances. Unfortunately, no follow-up is provided. The authors did not respond to repeated inquiries to asking whether they even obtained permission to use these quotes.

The quotes also have been trimmed of most details about their context that are available in original sources. Going to the original sources, we find the sources deliberately sampled people who were not service users.

Yup, people stripped of their identities are paraded out without the benefit of information that would render their experiences meaningful. Readers can’t independently assessment the uses to which the psychologist authors of Understanding Psychosis put these quotes.

What is not at issue is whether people with unusual experiences can get our attention when they talk about them. What is at issue is that a group of professionals take these quotes out of context and insist that they be accepted as the primary basis for – as their title states – our understanding of psychosis and schizophrenia.

Some of the quotes come from sources like

Jackson, L., Hayward, M. & Cooke, A. (2011). Developing positive relationships withvoices: A preliminary grounded theory. International Journal of Social Psychiatry, 57(5), 487–495.

Freeman, D., Garety, P.A., Bebbington, P.E., Smith, B., Rollinson, R., Fowler, D. et al. (2005). Psychological investigation of the structure of paranoia in a non-clinical population. British Journal of Psychiatry, 186, 427–435.

Heriot-Maitland, C., Knight, M. & Peters, E. (2012). A qualitative comparison of psychotic-like phenomena in clinical and non-clinical populations. British Journal of Clinical Psychology, 51(1), 37–53.

Jackson et al report

Five men and seven women were recruited through local NHS services, community advertisement and the local branch of the Hearing Voices Network.

Freeman  et al report

An anonymous internet survey [was]… e-mailed the address of a website where they could take part in a survey of ‘everyday worries about others’.

Heriot-Maitland et report interviewing 12 participants, who reported “psychotic-like ‘out-of-the-ordinary’ experience (OOE) in the past five years.”

The quotes come from persons who are lucid enough to be recruited for small studies of , highly selected articulate persons. They certainly don’t display the distorted thought and behavior disorder and simple incoherence of many people with acute and chronic schizophrenia.

I agree with the Understanding Psychosis authors that few people who have ‘psychotic-like’ experiences meet criteria for a diagnosis of schizophrenia. But should we accept a carefully cherry-picked and edited group of quotes as the basis for revising our understanding of people who do meet criteria?

A number of quotes sound like people who are high functioning and showing an unusual degree of  fantasy-proneness:

P 29 I work four days a week in a professional job; I own my own house and live happily with my partner and pets. Occasionally I hear voices – for example when I have been particularly stressed or tired, or I have seen visions after a bereavement. Knowing that many people hear voices and live well, and that some cultures see these experiences as a gift, helps me to never catastrophise or to worry that it may be the start of a breakdown. Although I am lucky that the experiences have never been as upsetting as some people’s, if someone had told me it was madness I could have got into a vicious cycle and struggled to get out.

Some of the quotes seem to represent clinically significant distress, but probably not psychosis or schizophrenia.

p 53 One thing that you might hear a lot about is that anxiety is a trigger of suspicious thoughts. I have never been that good at recognising my own anxiety. Quite a high level of anxiety is pretty normal for me. So normal that I wouldn’t normally do anything about it, but I now recognise that it sets the background for the expected potential threats in any situation, and so the suspicious thoughts and ideas of reference can pop right in there. I find people as having the most potential as a source of threat and because of that I am prone to suspicious thoughts about others. So now what I do is try to address the level of anxiety I feel in these situations. Adam

We’re not provided any information suggesting this suspiciousness is the psychotic symptom, paranoia.

P 43 After being almost killed by my ex-boyfriend when I was 16 I have had OCD. I have also developed paranoia about someone trying to kill me. If I have conflict with someone over anything I worry they are going to kill me or have someone come and kill me. I wake up worried someone is in my bedroom. I think about trying to be ready to protect myself if someone comes at me. I don’t think I would have this if I had not been traumatised half my life ago.  Josephine

Yale Professor Joan Cook and other colleagues and I recently published  a mixed method study of a national sample of psychotherapists providing residential treatment to veterans for posttraumatic stress disorder.  A number reported difficulties deciding whether the “voices” that some veterans describe represented schizophrenia or vivid re-experiencing symptoms consistent with posttraumatic stress syndrome, for which exposure therapy is indicated.

The authors of Understanding Psychosis express a clear disdain for making diagnostic distinctions. But,  it is important for clinicians to decide about the nature of clients’ distress in order to decide how to treat it. They best do so by formulating a hypothesis based on evidence tied to diagnoses, and then sympathetically probing.  Gradual exposure to past trauma would likely tame the distress of someone meeting criteria for PTSD. But this could prove absolutely terrifying and decompensating for someone whom additional information suggested a diagnosis of psychosis. So clinicians have to have some evidence-based ideas to probe and make decisions or proceed blindly.

Some quotes probably refer to brief psychotic reactions. Responding to Understanding Psychosis, Allan Frances noted

Brief psychosis is considered a mental disorder, but it is just a transient one with excellent prognosis and no reason to expect long-term impairment. The symptoms emerge suddenly in response to stress and usually disappear just as suddenly (especially if the stress is removed), often never to reappear. This is common in many cultures, and I have seen it fairly often in college students away from home for the first time, in travelers in strange lands, and in people who have had something terrible happen to them. Antipsychotic medicine is needed only briefly, if at all.

Quotes were selected to fit the authors’ conviction that what other professionals call psychosis or schizophrenia is an understandable reaction to life events. But if we go to the larger literature, the associations between adverse experiences and psychosis, even in a meta-analysis of one of the authors of Understanding Psychosis, are not large enough that would suggest such strong causality.  Adverse experiences are linked to lots of negative outcomes, but generally do not lead to psychosis or schizophrenia, even if there is a significant, but not overwhelming correlation.

Understanding Psychosis is not a transparent, systematic review of available evidence. Authors are mustering quotes to fit their preconceived notions. And leaving out quotes and details that don’t fit.

American psychiatrist Bernard “Barney” Carroll slammed the arrogant response of President of the American Psychiatric Association President  Jeffrey Lieberman to media coverage of Understanding Psychosis. Barney called it over-the-top” and a “disservice to psychiatry.” Yet, this was not before he nailed the report for its “domesticating psychosis”:

Hallucinations become the experience of hearing voices; delusions become the experience of unusual beliefs; paranoid thinking becomes the experience of anxiety – never mind that the great majority of patients with clinical anxiety disorders are not at all paranoid in the way that psychotic patients are. They also make much of the fact that milder forms of these “experiences” are common in the general population – as are milder forms of many clearly medical symptoms. In short, they fail to acknowledge the state transition that demarcates mild or prodromal symptoms from outright psychotic illness.

… The BPS document fails adequately to convey the range of symptoms and associated behaviors in psychosis/schizophrenia. Even when these are mentioned, they are not addressed in a way that matches their clinical salience. Thus, decompensating psychotic crises are discussed unhelpfully in the framework of poor sleep habits. Acute inpatient psychiatric units are discussed in a patronizing way and are faulted as being unhelpful for some patients – never mind their rescue function. Catatonia as a common feature is not acknowledged. Psychotic terror and panic are not acknowledged. Formal thought disorder with truly crazy speech is not acknowledged.

A disclosure of my past.

I’m struck by the huge gap between the clear, articulate statements in the quotes provided in Understanding Psychosis and the incoherent mumbling and sometimes raging of people who are acutely psychotic.  I wonder how many of the authors have ever tried to conduct an interview with someone in that state.

cowboy entering belgium-1-page-001My clinical training involved six years of live supervision at the Mental Health Institute (MRI)provided by professionals widely recognized for their innovative work in analyzing the communication of persons considered as having schizophrenia –  Paul Watzlawick, John Weakland, and Richard Fisch – although they would have objected to that diagnostic label.

At the time, I probably was more anti-diagnosis than many of the authors of Understanding Psychosis are today. But then as Director of Research at Mental Research Institute, I witnessed the disaster of its Soteria Project. I’ll leave that for another time, but Wikipedia states

The Soteria project was admired by many professionals around the world who aspired to create mental health services based on a social, as opposed to a medical, model. It was also heavily criticized as irresponsible or ineffective. The US Soteria Project closed as a clinical program in 1983 due to lack of financial support, although it became the subject of research evaluation with competing claims and analysis. Second generation US successors to the original Soteria house called Crossing Place is still active, although more focused on medication management.

While Paul, John, and Dick were widely recognized for their work analyzing communication with severely disturbed persons, they operated with a sense that at some point the disturbance of thought and behavior could became too much to carry on a discussion. And talking to highly disturbed persons, they knew not to take what was being said literally.

Who was selected for inclusion in Understanding Psychosis and who was excluded and left silent?

Many patients with acute and chronic psychosis are essentially nonverbal and cannot communicate their distress. Sure, they can’t provide coherent quotes for the psychologists who assembled Understanding Psychosis, but it is irresponsible for those psychologists to pretend these people don’t exist or that the quotes they assembled represent their best interest.

Many patients who meet criteria for schizophrenia will times be unable to take care of themselves or to make basic decisions.  The burden of caring and decision-making will fall on family members if they are available. The alternative for persons with schizophrenia is to become homeless or go to jail or prisons because more appropriate beds and hospitals are not available. Nowhere in Understanding Psychosis are we reminded that persons with schizophrenia sometimes need sanctuary in hospitals.

Nowhere are we reminded that 10% of persons with schizophrenia will die by suicide. There is recent evidence that psychotic people may account for nearly 1/3 of suicide attempts with intent to die.

If I were a family member of someone with schizophrenia, I would be damn angry at the gap between the quotes in Understanding Psychosis what I knew about the person for whom I had to provide care. I’d also be angry that no one in my situation had been invited to participate as a stakeholder.

Psychologists in search of opportunities to work with YAVIS clients

purchase of friendshipThe carefully selected quotes suggest people who would be more satisfying to work with than many persons with psychosis and schizophrenia. Reading them, I was immediately reminded of William Schofield['s  50-year-old book Psychotherapy: The Purchase of Friendship in which he lamented the strong tendency of mental health professionals wanting to work with the YAVIS: clients who are young, attractive, verbal, intelligent, and successful.  One of the authors of Understanding Psychosis also co-authored the widely misrepresented Lancet study of cognitive behavioral therapy for psychosis and could tell us how difficult and ineffective  it was doing  therapy in  that study with the older patients who had more psychotic  episodes.

Despite the authors of the Lancet study having distanced themselves from earlier claims showed cognitive therapy had effects equivalent to antipsychotic medication, authors of Understanding Psychosis persist in making the claim to service users:

It would also appear that CBT can bring comparable benefits even when people choose not to take medication.

As we would expect from recommendations produced by tightly knit groups representing single professions, Understanding Psychosis is a bid for more resources for its authors to work with clients with whom they want to work.  But like any policy recommendations, we need to examine the evidence and look at where those resources would come.

Please click to enlarge

Please click to enlarge

I’ll leave that discussion to another blog post, but take a look at the graph on the left. It represents the dramatic shift in resources from inpatient beds to outpatient treatment settings. The profoundly disturbed persons who need those beds would undoubtedly be less suitable for the conversations that the Understanding Psychosis psychologists want to be having. The long term reduction in inpatent services represents not so much deinstitutionalization as transinstutionalization.  A lack of those beds means that persons in need of them are being relegated to jails and prisons. In the United States, the Los Angeles jails represent the largest mental health treatment facility in the United States and the conditions for the severely disturbed are abominable. Similar situations hold in the UK.

An inpatient psychiatrist recently wrote in the New York Times:

We also need to rethink how we care for another group of vulnerable patients who have been just as disastrously disserved by policies meant to empower and protect them: the severely mentally disabled.

He went on:

We have worked to minimize the use of restraint and seclusion on my unit, but have seen the frequency of both skyrocket. Nearly every week staff members are struck or scratched by largely nonverbal patients who have no other way to communicate their distress. Attempting to soothe these patients monopolizes the efforts of a staff whose mission is to treat acute psychiatric emergencies, not chronic neurological conditions. Everyone loses.

Professor-Simon-Wessely-007Somebody in the UK should be speaking up for the inarticulate vulnerable persons with schizophrenia needing inpatient beds who are silenced and marginalized by the authors of Understanding Psychosis.  Where the hell is Simon Wessely when they need him?

Promoting an unrealistic view of schizophrenia?

If the authors of Understanding Psychosis were truly interested in providing authoritative information for persons with schizophrenia or psychosis, their family members, and professionals who come into contact with them, they would’ve provided the latest evidence about long-term course and outcome.

For instance, a key English study provides a 10 year follow-up individuals with a first episode of psychosis initially identified in either southeast London or Nottingham.

Morgan, C., Lappin, J., Heslin, M., Donoghue, K., Lomas, B., Reininghaus, U., … & Dazzan, P. (2014). Reappraising the long-term course and outcome of psychotic disorders: the AESOP-10 study. Psychological medicine, 44(13), 2713-2726.

At follow-up, of 532 incident cases identified, at baseline 37 (7%) had died, 29 (6%) had emigrated and eight (2%) were excluded. Of the remaining 458, 412 (90%) were traced and some information on follow-up was collated for 387 (85%). Most cases (265, 77%) experienced at least one period of sustained remission; at follow-up, 141 (46%) had been symptom free for at least 2 years. A majority (208, 72%) of cases had been employed for less than 25% of the follow-up period. The median number of hospital admissions, including at first presentation, was 2 [interquartile range (IQR) 1–4]; a majority (299, 88%) were admitted a least once and a minority (21, 6%) had 10 or more admissions. Overall, outcomes were worse for those with a non-affective diagnosis, for men and for those from South East London.Conclusions Sustained periods of symptom remission are usual following first presentation to mental health services for psychosis, including for those with a non-affective disorder; almost half recover.

Put differently, overall

12% (9% for non-affective) of our sample recovered within 6 months of contact with services and did not have a further episode, 20% (14% for non-affective) never had an episode lasting more than 6 months, and around 50% (40% for non-affective) had not experienced symptoms in the 2 years prior to follow-up.

And then there is the most recent comprehensive systematic review and meta-analysis.

Jääskeläinen, E., Juola, P., Hirvonen, N., McGrath, J. J., Saha, S., Isohanni, M., … & Miettunen, J. (2012). A systematic review and meta-analysis of recovery in schizophrenia. Schizophrenia Bulletin, sbs130.

We identified 50 studies with data suitable for inclusion. The median proportion (25%–75% quantiles) who met our recovery criteria was 13.5% (8.1%–20.0%). Studies from sites in countries with poorer economic status had higher recovery proportions. However, there were no statistically significant differences when the estimates were stratified according to sex, midpoint of intake period, strictness of the diagnostic criteria, duration of follow-up, or other design features. Conclusions: Based on the best available data, approximately, 1 in 7 individuals with schizophrenia met our criteria for recovery. Despite major changes in treatment options in recent decades, the proportion of recovered cases has not increased.

One in 7 people with schizophrenia meet criteria for recovery, and the portion has not increased in recent decades. Compare that with the unrealistically cheery assessment offered in Understanding Psychosis:

Even if people continue to hear voices or hold unusual beliefs, they may nevertheless lead very happy and successful lives. Sometimes a tendency to ‘psychosis’ can be associated with particular talents or abilities.

And

p 30 People who continue to have severe and distressing experiences may lead happy and successful lives in all other respects, such as work and relationships.

Sure, the authors of Understanding Psychosis keep reminding us of the cliché that everybody is different. But they are asking us to make clinical and policy decisions that are life altering for some people and could be life-ending for others. We can’t afford to ignore a larger body of relevant data.

In light of the data from long term follow-up studies, Understanding Psychosis should be seen as a cruel hoax perpetrated against more typical severely disturbed mental health service users, their family, and policymakers

Category: evidence-supported, mental health care, professional organizations, psychosis, psychotherapy, schizophrenia | Tagged , , , , | Leave a comment

Biomarker Porn: From Bad Science to Press Release to Praise by NIMH Director

Concluding installment of NIMH biomarker porn: Depression, daughters, and telomeres

Pioneer HPA-axis researcher Bernard “Barney” Carroll’s comment left no doubt about what he thought of the Molecular Psychiatry article I discussed in my last issue of Mind the Brain:

Where is the HPA axis dysregulation? It is mainly in the minds0@PubSubMain@NIHMS2@s@0@44595.html of the authors, in service of their desired narrative. Were basal cortisol levels increased? No. Were peak cortisol levels increased? They didn’t say. Was the cortisol increment increased? Only if we accept a p value of 0.042 with no correction for multiple comparisons. Most importantly, was the termination of the stress cortisol response impaired? No, it wasn’t (Table 3). That variable is a feature of allostasis, about which co-author Wolkowitz is well informed. Termination of the stress response is a crucial component of HPA axis regulation (see PubMed #18282566), and it was no different between the two groups. So, where’s the beef? The weakness of this report tells us not only about the authors’ standards but also about the level of editorial tradecraft on display in Molecular Psychiatry. [Hyperlink added]

You also can see my response to Professor Carroll in the comments.

I transferred  another  comment  to the blog from my Facebook wall. It gave me an opportunity to elaborate on why

we shouldn’t depend on small convenience samples to attempt to understand phenomena that must be examined in larger samples followed prospectively.

I explained

There are lots of unanswered questions about the authors’ sampling of adolescents. We don’t know what they are like when their mothers are not depressed. The young girls could also simply be reacting to environmental conditions contributing to their mother’s depression, not to their mother’s depression per se. We don’t know how representative this convenience sample is of other daughters of depressed mothers. Is it unusual or common that daughters of this age are not depressed concurrent with their mothers’ depression? What factors about the daughters, the mothers, or their circumstances determine that the mother and daughter depression does not occur at the same time? What about differences with him him dthe daughters of mothers who are prone to depression, but are not currently depressed?  We need to keep in mind that most biomarkers associated with depression are state dependent, not trait dependent. And these daughters were chosen because they are not depressed…

But with no differences in cortisol response, what are we explaining anyway?

The Molecular Psychiatry article provides an excellent opportunity to learn to spot bad

From  http://www.compoundchem.com/2014/04/02/a-rough-guide-to-spotting-bad-science/

From http://www.compoundchem.com/2014/04/02/a-rough-guide-to-spotting-bad-science/

science. I encourage interested readers to map what is said in that into the chart at the right.

This second installment of my two-part blog examines how the exaggerations and distortions of the article reverberate through a press release and then coverage in NIMH Director Thomas Insel’s personal blog.

The Stanford University press release headline is worthy of the trashy newspapers we find at supermarket checkouts:

Girls under stress age more rapidly, new Stanford study reveals

The press release says things that didn’t appear in the article, but echoes the distorted literature review of the article’s introduction in claiming well-established links between shortened telomeres, frequent infections in chronic disease and death that just are not there.

The girls also had telomeres that were shorter by the equivalent of six years in adults. Telomeres are caps on the ends of chromosomes. Every time a cell divides the telomeres get a little shorter. Telomere length is like a biological clock corresponding to age. Telomeres also shorten as a result of exposure to stress. Scientists have uncovered links in adults between shorter telomeres and premature death, more frequent infections and chronic diseases.

From http://news.stanford.edu/news/2014/october/telomeres-depression-girls-10-28-2014.html

From http://news.stanford.edu/news/2014/october/telomeres-depression-girls-10-28-2014.html

And the claim of “the equivalent of six years” comes from direct quote from obtained from senior author Professor Ian Gotlib.

“It’s the equivalent in adults of six years of biological aging,” Gotlib said, but “it’s not at all clear that that makes them 18, because no one has done this measurement in children.”

Dr. Gotlib  seems confused himself about what he mean by the 10 to 14-year-old girls having aged an additional six years. Does he really think that they are now 18? If so in what way? What could he possibly mean – do they look six years older than age matched controls? That would be really strange if they did.

I hope he lets us know when he figures out what he were saying, but he shouldn’t have given the statement to the Stanford press officer unless he was clear what he meant.

The press release noted that Dr. Gotlib had already moved on to intervention studies designed to prevent telomere shortening these girls.

In other studies, Gotlib and his team are examining the effectiveness of stress reduction techniques for girls. Neurofeedback and attention bias training (redirecting attention toward the positive) seem promising. Other investigators are studying techniques based on mindfulness training.

That’s a move based on speculation, if not outright science-fiction. Neurofeedback has some very preliminary evidence for effectiveness in treating current depression, but I would like to see evidence that it has any benefit for preventing depression in young persons who have never been depressed

neurofeedbackGotlib’s claims play right into popular fantasies about rigging people up with some sort of apparatus that changes their brain. But everything changes the brain, even reading this blog post. I don’t think that reading this blog post has any less evidence for preventing later depression than neurofeedback. Nonetheless, I’m hoping  that my blogging implants a healthy dose of skepticism in readers’ brains so that they are immunized against further confusion from exposure to such press releases. For an intelligent, consumer oriented discussion of neurofeedback, see Christian Jarrett’s

Read this before paying $100s for neurofeedback therapy

Attention bias training is a curious choice. It is almost as trendy as neurofeedback, but would it work?  We have the benefit of a systematic review and recent meta-analysis that suggests a lack of evidence for attention bias training in  treating depression and no evidence for preventing it. If it’s ineffectual in treating depression, how could we possibly expect it to prevent depression? Evidence please!

Let’s speculate about the implications if the authors found the cortisol differences between the daughters of the depressed mothers and daughters of controls that they had hypothesized but did not find. What then could have been done for these young girls? Note that the daughters of depressed mothers were chosen because they were functioning well, not currently depressed themselves. Just because they were different from the control girls would not necessarily indicate that any cortisol variables were in the abnormal range. Cortisol levels are not like blood pressure – we cannot specify a level below which cortisol levels have to be brought down for better health and functioning.

Note also that these daughters were selected on the basis of their mothers being depressed and that could mean the daughters themselves were facing a difficult situation. We can’t make the mother-bashing assumption that their mother’s depression was inflicting stress on them. Maybe any psychobiological stress response that was evident was due to the circumstances that led to the depression of their mother. We don’t know enough to specify what levels of cortisol variables would be optimal and consistent with good coping with the situation – we let even specify what is normal. And we don’t know how the daughters would recover from any abnormalities without formal treatment when their circumstances changed.

Bottom line is that these investigators did not get the results they hypothesized. Even if they had, results would not necessarily to lead to clinical applications.

Nonetheless, the director of NIMH saw fit to single this paper out or maybe he was just picking up on the press release.

my blogThomas Insel’s Personal Blog: Depression, Daughters, and Telomeres.

Thomas Insel’s Director’s Blog starts by acknowledging that there are no genetic or imaging markers predicting risk for depression, but research by Stanford Psychology Professor Ian Gotlib and colleagues in Molecular Psychiatry is “worth watching.”

Insel describes Gotlib’s “longitudinal” research as following depressed mothers’ early adolescent daughters.

The young girls have not yet developed depression, but 60 percent will become depressed by the age of 18.

I can find no basis in the article for Insel’s claim that Gotlib has found 60 per cent of these girls will be depressed by age 18. The estimate seems exaggerated, particularly given the case mix of mothers of these girls. It appears that some or most of the mothers were drawn from the community. We cannot expect severe course and biological correlates of depression that we would expect from a more inpatient sample.

Searching the papers coming out of this lab, I could only find one study involving a 30 month follow-up of 22 daughters of depressed mothers in the same age range as the sample in the Molecular Psychiatry article. That’s hardly a basis for the strong claim of 60% becoming depressed by 18.

Insel embellishes the importance of differences in telomere length. He perpetuates the illusion that we can be confident that differences in telomere length suggest these girls were experiencing accelerated aging and what have high risk for disease when the girls reached middle and late age. Without the backing of data from the paper or the existing literature, Insel zeros in on

Troubling early sign of risk for premature biological aging and possibly age-related chronic diseases, such as cardiovascular disease. Investigating the cause and timing of decreased telomere length—to what extent it may result from abnormalities in stress responses or is genetically influenced, for example—will be important for understanding the relationship between cellular aging, depression, and other medical conditions.

Insel ponders how such young, healthy girls could possibly show signs of aging. According to him the answer is not clear, but it might be tied to the increased stress reactivity these girls show in performing laboratory tasks.

But as Professor Caroll noted, the study just does not much evidence of “increased stress reactivity.”

neurofeedback2jpgNonetheless, Insel indicates that Gotlib’s next step is

Using neurofeedback to help these girls retrain their brain circuits and hopefully their stress responses. It will be a few years before we will know how much this intervention reduces risk for depression, but anything that prevents or slows the telomere shortening may be an early indication of success.

It’s interesting that Insel sidestepped the claim in the press release that Gotlib was trying out a cognitive behavioral intervention to affect stress reactivity. Instead he presents a fanciful notion that neural feedback will somehow retrain these girls’ brain circuits and reduce their stress response throughout their time at home and prevent them getting depressed by their mother’s depression.

Oh, if that were only so: Insel would be vindicated in his requiring for funding that researchers get down to basic mechanisms and simply bypass existing diagnoses with limited reliability, but at least some ties to patients’ verbal reports of why they are seeking treatment. In his world of science fiction, patients, or at least these young girls, which come in to have their brains retrained to forestall the telomere shortening that is threatening them not only with becoming depressed later, but with chronic diseases and middle and late life and early death.

So, let’s retrace what was said in the original Molecular Psychiatry article to what was claimed in the Stanford University press release and what was disseminated in the social media of Dr. Insel’s personal blog. Authors’ spin bad science in a peer-reviewed article. They collaborate with their university’s press relations department by providing even more exaggerated claims. And Dr. Insel’s purpose is served by simply passing them on and social media.

There’s a lot in Dr. Insel’s Personal Blog to disappoint and even outrage

  • Researchers  seeking guidance for funding priorites.
  • Clinicians in the trenches needing to do something now to deal with the symptoms and simple misery that are being presented to them.
  • Consumers looking for guidance from the Director of NIMH as to whether they should be concerned about their daughters and what they should do about it.

A lot of bad science and science fiction is being served to back up false promises about anything likely to occur in our lifetimes, if ever.

promising treatmentTaxpayers need to appreciate where Dr. Insel is taking funding of mental health with research. He will no longer fund grants that will explore different psychotherapeutic strategies for common mental health problems as they are currently understood – you know, diagnoses tied to what patients complain about. Instead he is offering a futuristic vision in which we no longer have to pay for primary care physicians or mental health clinicians spending time talking to patients about the problems in their lives. Rather, patients can bring in a saliva sample to assess the telomere length. They then can be rigged up to a videogame providing a social stress challenge. They will then be given neurofeedback and asked to provide another saliva sample. If the cortisol levels aren’t where they are supposed to be, they will come back and get some more neurofeedback and videogames.

But wait! We don’t even need to wait until people develop problems in their lives. We can start collecting spit samples when they are preteens and head off any problems developing in their life with neural feedback.

Presumably all this could be done by technicians who don’t need to be taught communication skills. And if the technicians are having problems, we can collect spit samples from them and maybe give them some neurofeedback.

Sure, mild to moderate depression in the community is a large and mixed grouping. The diagnostic category major depression loses some of its already limited reliability and validity when applied to this level of severity. But I still have a lot more confidence in this diagnosis than relying on some unproven notions about treating telomere length and cortisol parameters in people who do not currently complain about mental health or their circumstances. And the lamer notion that this can be done without any empathy or understanding.

It’s instructive to compare what Insel says in this blog post to what he recently said in another post.

He acknowledged some of the serious barriers to the development of valid, clinically useful biomarkers:

Patients with mental disorders show many biological abnormalities which distinguish them from normal volunteers; however, few of these have led to tests with clinical utility. Several reasons contribute to this delay: lack of a biological ‘gold standard’ definition of psychiatric illnesses; a profusion of statistically significant, but minimally differentiating, biological findings;‘approximate replications’ of these findings in a way that neither confirms nor refutes them; and a focus on comparing prototypical patients to healthy controls which generates differentiations with limited clinical applicability. Overcoming these hurdles will require a new approach. Rather than seek biomedical tests that can ‘diagnose’ DSM-defined disorders, the field should focus on identifying biologically homogenous subtypes that cut across phenotypic diagnosis—thereby sidestepping the issue of a gold standard.

All but the last sentence could have been part of a negative review of the Molecular Psychiatry article or the grant that provided funding for it. But the last sentence is the kind of nonsense that a director of NIMH can lay on the and research community and expect it to be reflected in their grant applications.

But just what was the theme of this other blog post from Dr. Insel? P-hacking and the crisis concerning results of biomedical research not being consistently reproducible.

The relentless quest for a significant “P” value is only one of the many problems with data analysis that could contribute to the reproducibility problem. Many mistakenly believe that “P” values convey information about the size of the difference between two groups. P values are actually only a way of estimating the likelihood that the difference you observe could have occurred by chance. In science, “significance” usually means a P value of less than 0.05 or 1 in 20, but this does not mean that the difference observed between two groups is functionally important. Perhaps the biggest problem is the tendency for scientists to report data that have been heavily processed rather than showing or explaining the details. This suggests one of the solutions for P-hacking and other problems in data analysis: provide the details, including what comparisons were planned prior to running the experiment.

Maybe because Insel is Director of NIMH, he doesn’t expect anybody to call him on the contradictions in what he is requesting. In the p-hacking blog post, he endorsed a call to action to address the problem of a lot of federal money being wasted on research that can’t lead to improvements in the health and well-being of the population because the research is simply unreliable and depends on “heavily processed” data for which investigators don’t provide the details. Yet in the Depression, Daughters, and Telomeres post he grabs an outrageous example of this being done and tells the research community he wants to see more of it.

 

porn

Category: biomarkers, HPA Axis, hype, maternal depression, mental health care, stress | Tagged , , , , | 3 Comments

NIMH Biomarker Porn: Depression, Daughters, and Telomeres Part 1

Does having to cope with their mother’s depression REALLY inflict irreversible damage on daughters’ psychobiology and shorten their lives?

telomerejpg

Telomere

A recent BMJ article revived discussion of responsibility for hyped and distorted coverage of scientific work in the media. The usual suspects, self-promoting researchers, are passed over and their University press releases are implicated instead.

But university press releases are not distributed without authors’ approval.  Exaggerated statements in press releases are often direct quotes from authors. And don’t forget the churnaling journalists and bloggers who uncritically pass on press releases without getting second opinions.  Gary Schwitzer remarked:

Don’t let news-release-copying journalists off the hook so easily. It’s journalism, not stenography.

In this two-part blog post, I’ll document this process of amplification of the distortion of science from article to press release to subsequent coverage. In the first installment, I’ll provide a walkthrough commentary and critique of a flawed small study of telomere length among daughters of depressed women published in the prestigious Nature Publishing Group journal, Molecular Psychiatry. In the second, I will compare the article and press release to media coverage, specifically the personal blog of NIMH Director Thomas Insel.

whackI warn the squeamish that I will whack some bad science and outrageous assumptions with demands for evidence and pelt the study, its press release, and Insel’s interpretation with contradictory evidence.

I’m devoting a two-part blog to this effort. Bad science with misogynist, mother bashing assumptions is being touted by the  Director of NIMH as an example to be followed. When he speaks, others pay attention because he sets funding priorities. Okay, Dr. Insel, we will listen up, but we will do so skeptically.

A paper that shares an author with the Molecular Psychiatry paper was criticized by Daniel Engber for delivering

A mishmash of suspect stats and overbroad conclusions, marshaled to advance a theory that’s both unsupported by the data and somewhat at odds with existing research in the field.

The criticism applies to this paper as well.

But first, we need to understand some things about telomere length…

What is a Telomere?

Telomeres are caps on the ends of every chromosome. They protect the chromosome from losing important genes or sticking to other chromosomes. They become shorter every time the cell divides.

I have assembled some resources in an issue of Science-Based Medicine:

Skeptic’s Guide to Debunking Claims about Telomeres in the Scientific and Pseudoscientific Literature

As I say in that blog, there are many exaggerated and outright pseudoscientific claims about telomere length as a measure of “cellular aging” and therefore how long we’re going to live.

I explain the concepts of biomarker and surrogate endpoint, which are needed to understand the current fuss about telomeres. I show why the evidence is against routinely accepting telomere length as a biomarker or surrogate endpoint for accelerated aging and other health outcomes.

I note

  • A recent article in American Journal of Public Health claimed that drinking 20soda kills ounces of carbonated (but not noncarbonated) sugar-sweetened drinks was associated with shortened telomere length “equivalent to an approximately 4.6 additional years of aging.” So, effects of drinking soda on life expectancy is equivalent to what we know about smoking’s effect.
  • Rubbish. Just ignore the telomere length data and directly compare the effects of drinking 20 ounces soda to the effects of smoking on life expectancy. There is no equivalence. The authors confused differences in what they thought was a biomarker with differences in health outcomes and relied on some dubious statistics. The American Journal of Public Health soda study was appropriately skewered in a wonderful Slate article, which I strongly recommend.
  • Claims are made for telomere length as a marker for effects of chronic stress and risk of chronic disease. Telomere length has a large genetic component and is correlated with age. When appropriate controls are introduced, correlation among telomere length, stress, and health outcomes tend to disappear or get sharply reduced.
  • A 30-year birth cohort study did not find an association between exposure to stress and telomere length.
  • Articles from a small group of investigators claim findings about telomere lengths that do not typically get reproduced in larger, more transparently reported studies by independent groups. This group of investigators tends to have or have had conflicts of interest in marketing of telomere diagnostic services, as well as promotion of herbal products to slow or reverse the shortening of telomere length.
  • Generally speaking, reproducible findings concerning telomere length require large samples with well-defined phenotypes, i.e., individuals having well-defined clinical presentations of particular characteristics, and we can expect associations to be small.

Based on what I have learned about the literature concerning telomere length, I would suggest

  • Beware of small studies claiming strong associations between telomere length and characteristics other than age, race, and gender.
  • Beware of studies claiming differences in telomere length arising in cross-sectional research or in the short term if they are not reproduced in longitudinal, prospective studies.

A walk-through commentary and critique of the actual article

Gotlib, I. H., LeMoult, J., Colich, N. L., Foland-Ross, L. C., Hallmayer, J., Joormann, J., … & Wolkowitz, O. M. (2014). Telomere length and cortisol reactivity in children of depressed mothers. Molecular Psychiatry.

Molecular Psychiatry is a pay-walled journal, but a downloadable version of the article is available here.

Conflict of Interest Statement

The authors report no conflict of interest. However, in the soda article published December 2014, one of the authors of the present paper, Jun Lin disclosed being a shareholder in Telomere Diagnostics, Inc., a telomere measurement company. Links at my previous blog post take you to “Telomeres and Your Health: Get the Facts” at the website of that company. You find claims that herbal products based on traditional Chinese medicine can reduce the shortening of telomeres.

Jun Lin has a record of outrageous claims. For instance, in another article, that normal women whose minds wander may be losing four years of life, based on the association between self-reported mind wandering and telomere length. So, if we pit this claim against what is known about the effects of smoking on life expectancy, women can extend their lives almost as much by better paying attention as from quitting smoking.

Hmm, I don’t know if we have undeclared conflict of interest here, but we certainly have a credibility problem.

The Abstract

Past research shows distorted and exaggerated media portrayals of studies are often already evident in abstracts of journal articles. Authors engage in a lot of cherry picking and spin results to strengthen the case their work is innovative and significant.

The opening sentence of the abstract to this article is a mashup of wild claims about telomere length in depression and risk for physical illnesses. But I will leave commenting until we reach the introduction, where the identical statement appears with elaboration and a single reference to one of the author’s work.

The abstract goes on to state

Both MDD and telomere length have been associated independently with high levels of stress, implicating dysregulation of the hypothalamic-pituitary-adrenal (HPA) axis and anomalous levels of cortisol secretion in this relation.

hpa useWhen I showed this to a pioneer in the study of the HPA axis, he remarked:

If you can find coherence in this from the Abstract you are smarter than I am…The phrase dysregulation of the HPA axis has been used to support more hand waving than substance.

The abstract ends with

This study is the first to demonstrate that children at familial risk of developing MDD are characterized by accelerated biological aging, operationalized as shortened telomere length, before they had experienced an onset of depression; this may predispose them to develop not only MDD but also other age-related medical illnesses. It is critical, therefore, that we attempt to identify and distinguish genetic and environmental mechanisms that contribute to telomere shortening.

This breathless editorializing about the urgency of pursuing this line of research is not tied to the actual methods and results of the study. “Accelerated biological aging” and “predispose to develop… other age-related medical illnesses” is not a summary of the findings of the study, but only dubious assumptions.

Actually, the evidence for telomere length as a biomarker for aging is equivocal and does not meet American Federation of Aging Research criteria.  A large scale prospective study did not find that telomere length predicted onset of diabetes or cardiovascular disease.

And wait to when we examine whether the study had reproducible results concerning either shorter telomeres and depression or telomeres being related to cortisol reactivity.

The introduction

The 6-paragraph introduction packs in a lot of questionable assumptions backed by a highly selective citation of the literature.

A growing body of research demonstrates that individuals diagnosed with major depressive disorder (MDD) are characterized by shortened telomere length, which has been posited to underlie the association between depression and increased rates of medical illness, including cardiovascular disease, diabetes, metabolic syndrome, osteoporosis and dementia (see Wolkowitz et al.1 for a review).

Really? A study co-authored by Wolkowitz and cited later in the introduction actually concluded

telomere shortening does not antedate depression and is not an intrinsic feature. Rather, telomere shortening may progress in proportion to lifetime depression exposure.

“Exposure” = personal experience being depressed. This would seem to undercut the rationale for examining telomere shortening in young girls who have not yet become depressed.

But more importantly, nether the Molecular Psychiatry article nor the Wolkowitz review acknowledge the weakness of evidence for

  • Depression being characterized by shortened telomere length.
  • The association of depression and medical illness in older persons representing a causal role for depression that can be modified by or prevention or treatment of depression in young people.
  • Telomere length observed in the young underlying any association between depression and medical illnesses when they get old.

Wolkowitz’s “review” is a narrative, nonsystematic review. The article assumes at the outset that depression represents “accelerated aging” and offers a highly selective consideration of the available literature.

In neither it nor the Molecular Psychiatry article we told

  • Some large scale studies with well-defined phenotypes fail to find associations between telomeres and depressive disorder or depressive symptoms. One large-scale study co-authored by Wolkowitz found weak associations between depression and telomere length too small to be detected in the present small sample. Any apparent association may well spurious.
  • The American Heart Association does not consider depression as a (causal) risk factor for cardiovascular disease, but as a risk marker because of a lack of the evidence needed to meet formal criteria for causality. Depression after a heart attack predicts another heart attack. However, our JAMA systematic review revealed a lack of evidence that screening cardiac patients for depression and offering treatment reduces their likelihood of having another heart attack or improves their survival. An updated review confirmed our conclusions.
  • The association between recent depressive symptoms and subsequent dementia is evident with very low level of symptoms, suggesting that it reflects residual confounding and reverse causation  of depressive symptoms with other risk factors, including poor health and functioning. I published a commentary in British Medical Journal  that criticized  claim that we should begin intervening for even low symptoms of depression in order to prevent dementia. I suggested that we would be treating a confound and it would be unlikely to make a difference in outcomes.

I could go on. Depression causally linked to diabetes via differences in telomere length? Causing osteoarthritis? You gotta be kidding. I demand quality evidence. The burden of evidence is on anyone who makes such wild claims.

Sure, there is lots of evidence that if people have been depressed in the past, they are more likely to get depressed again when they have a chronic illness. And their episodes of depression will last longer.

In general, there are associations between depression and onset and outcome of chronic illness. But the simple, unadjusted association is typically seen at low levels of symptoms, increases with age and accumulation of other risk factors and other physical co-morbidities. People who are older, already showing signs of illness, or who have poor health-related behaviors tend to get sicker and die. Statistical control for these factors reduces or eliminates the apparent association of depressive symptoms with illness outcomes. So, we are probably not dealing with depression per se.  If you are interested in further discussion of this see my slide presentation, see

Negative emotion and health: why do we keep stalking bears, when we only find scat in the woods?

I explain risk factors (like bears) versus risk markers (like scat) and why shooting scat does not eliminate the health risk posed by bears,.

I doubt few people familiar with the literature believe that associations among telomeres and depression, depression and the onset of chronic illness, and telomeres and chronic illness are such that a case could be made for telomere length in young girls being importantly related to physical disease in their mid and late life. This is science fiction being falsely presented as evidence-based.

The authors of the Molecular Psychiatry paper are similarly unreliable when discussing “dysregulation of the hypothalamic-pituitary-adrenal (HPA) axis and anomalous levels of cortisol secretion.” You would think that they are referring to established biomarkers for risk of depression. Actually, most biological correlates of depression are modest, nonspecific to depression, and state, not trait-related – limited to when people are actually depressed.

MDD and ND [nondepressed] individuals exhibited similar baseline and stress cortisol levels, but MDD patients had much higher cortisol levels during the recovery period than their ND counterparts.

We did not find the expected main effects of maternal depression on children’s cortisol  reactivity.

  • They misrepresent a directly relevant study that examined cortisol secretion in the saliva of adolescents as a predictor of subsequent development of depression.  It actually found no baseline measure of cortisol measures predicted development of depression except cortisol awakening response.

In general, cortisol secretion is more related to stress than to clinical depression. One study concluded

The hypothalamic—pituitary—adrenal axis is sensitive to social stress but does not mediate vulnerability to depression.

depressed girlWhat is most outrageous about the introduction, however, is the specification of the pathway between having a depressed mother and shortened telomere length:

The chronic exposure of these children to this stress as a function of living with mothers who have experienced recurrent episodes of depression could represent a mechanism of accelerated biologic aging, operationalized as having shorter telomere length.

Recognize the argument that is being set up: having to deal with the mothers’ depression is a chronic stressor for the daughters, which sets up irreversible processes before the daughters even become depressed themselves, leading to accelerated aging, chronic illness, and early death. We can ignore all the characteristics, including common social factors, that the daughter share with their mothers, that might be the source of any daughters’ problems.

This article is a dream paper for the lawyers for men seeking custody of their children in a divorce: “Your honor, sole custody for my client is the children’s only hope, if it is not already too late. His wife’s depression is irreversibly damaging the children, causing later sickness and early death. I introduced as evidence of an article by Ian Gotlib that was endorsed by the Director of the National Institute of Mental Health…

Geraldine Downey and I warned about this trap in a classic review, children of depressed parents, cited 2300 times according to Google Scholar and still going strong. We noted that depressed mothers and their children share a lot of uncharted biological, psychological, and environmental factors. But we also found that among the strongest risk factors for maternal depression are marital conflict, other life events generated by the marriage and husband, and a lack of marital support. These same factors could contribute to any problems in the children. Actually, the husband could be a source of child problems. Ignoring these possibilities constitutes a “consistent, if unintentional, ‘mother-bashing’ in the literature.”

The authors have asked readers to buy into a reductionist delusion. They assume some biological factors in depression are so clearly established that they can serve as biomarkers.  The transmission of any risk for depression associated with having a depressed mother is by way of irreversible damage to telomeres. We can forget about any other complex social and psychological processes going on, except that the mothers’ depression is stressing the daughters and we can single out a couple of biological variables to examine this.

Methods

The Methods lacks basic details necessary to evaluate the appropriateness of what was done and the conclusions drawn from any results. Nonetheless, there is good reason to believe that we are dealing with a poorly selected sample of daughters from poorly selected mothers.

We’re not told much about the mothers except that they have experienced recurrent depression during the childhood of the daughters. We have to look to other papers coming out of this research group to discover how these mothers were probably identified. What we see is that they are a mixed group, in part drawn from outpatient settings and in part from advertisements in the community.

Recall that identification of biological factors associated with depression requires well-defined phenotypes. The optimal group to study would be patients with severe depression. We know that depression is highly heterogeneous and that “depressed” people in the community who are not in specialty treatment are likely to just barely meet criteria. We are dealing with milder disorder that is less likely to be characterized by any of the biological features of more severe disorder. Social factors likely play more of a role in their misery. In many countries, medication would not be the first line of treatment.

Depression is a chronic, remitting, recurrent disorder with varying degrees of severity of overall course and in particular episodes. It has its onset in adolescence or early adulthood. By the time women have daughters who are 10 to 14 years old, they are likely to have had multiple episodes. But in a sample selected from the community, these episodes may have been mild and not necessarily treated, nor even noticeable by the daughters. The bottom line is we should not be too impressed with the label “recurrent depression” without better documentation of the length, severity, and associated impairment of functioning.

Presumably the depressed mothers in the study were selected because they were currently depressed. That makes it difficult to separate out enduring factors in the mothers and their social context versus those that are tied to the women currently being depressed. And because we know that most biological factors associated with depression are state dependent, we may be getting a skewed picture of the biology of these women – and their daughters, for that matter – then at other times.

Basically, we are dealing with a poorly selected sample of daughters from a poorly selected sample of mothers with depression. The authors are not telling us crucial details that we need to understand any results they get. Apparently they are not measuring relevant variables and have too a small sample to apply statistical controls anyway.As I said about another small study making claims for a blood test for depression, these authors are

Looking for love biomarkers in all the wrong places.

Recall that I also said that results from small samples like this one often conflict with results from larger, epidemiologic studies with larger samples and better defined phenotypes. I think we can see the reasons why developing here. The small sample consist only of daughters who have a depressed mother, but who have not yet become depressed themselves and have low scores on a child depression checklist. Just how representative is the sample? What proportion of daughters this age of depressed women would meet these criteria? How are they similar or different from daughters who have already become depressed? Do the differences lie in their mothers or in the daughters or both? We can’t address any of these questions, but they are highly relevant. That’s why we need more larger clinical epidemiologic studies and fewer small studies of poorly defined samples. Who knows what selection biases are operating?

Searching the literature for what this lab group was doing in other studies in terms of mother and daughter recruitment, I came across a number of small studies of various psychological and psychobiological characteristics of the daughters. We have no idea whether the samples are overlapping or distinct. We have no idea about how the results of these other modest studies confirm or contradict results of the present one. But integrating their results with the results of the present study could have been a start in better understanding it.

As noted in my post at Science Based Medicine, we get a sense of the methods section of the Molecular Psychiatry article of unreliability in single assessments of telomeres. Read the description of the assay of telomere length in the article to get a sense of the authors having to rely on multiple measurements, as well as the unreliability of any single assessment. Look at the paragraph beginning

To control for interassay variability…

This description reflects the more general problems in the comparability of assessment of telomeres across individuals, samples, and laboratories problems that, that preclude recommending telomere length as a biomarker or surrogate outcome with any precision.

Results and Interpretation

As in the methods, the authors fail to supply basic details of the results and leave us having to trust them. There is a striking lack of simple descriptive statistics and bivariate relations, i.e., simple correlations. But we can see signs of unruly, difficult to tame data and spun statistics. And in the end, there are real doubts that there is any connection in these data between telomeres and cortisol.

The authors report a significant difference in telomere length between the daughters of depressed women versus daughters in the control group. Given how the data had to be preprocessed, I would really like to see a scatter plot and examine the effects of outliers before I came to a firm conclusion. With only 50 daughters of depressed mothers and 40 controls, differences could have arose from the influence of one or two outliers.

We are told that the two groups of young girls did not differ in Tanner scores, i.e., self-reported signs of puberty. If the daughters of depressed women had indeed endured “accelerated aging,” would it be reflected in Tanner scores? The authors and for that matter, Insel, seem to take quite literally this accelerated aging thing.

I think we have another seemingly large difference coming from a small sample that is statistically improbable to yield such a difference, given past findings. I could be convinced by these data of group differences in telomere length, but only if findings were replicated in an independent, adequately sized sample. And I still would not know what to make of them.

The authors fuss about  anticipating a “dysregulation of the hypothalamic-pituitary-adrenal (HPA) axis and anomalous levels of cortisol secretion.” They indicate that the cortisol data was highly skewed and had to be tamed by winsorizing, i.e., substituting arbitrary values for outliers. We are not told for how many subjects this was done or from which group they came. The authors then engaged in some fancy multivariate statistics, “a piecewise linear growth model to fit the quadratic nature of the [winsorized] data.”  We need to keep in mind that multilevel modeling is not a magic wand to transform messy data. Rather, it involves some assumptions that need to be tested and not assumed. We get no evidence of the assumptions being tested and the small sample sizes is such that they could not be reliably tested.

The authors found no differences in baseline cortisol secretion. Moreover, they found no differences in distress recovery for telomere length, group (depressed versus nondepressed mother), or group by telomere interaction. They found no effect for group or group by telomere interaction, but they did find a just significant (p< .042) main effect for telomere length on cortisol reactivity. This would not to seem to offer much support for a dysregulation of the HPA axis or anomalous levels of cortisol secretion associated with group membership (having a depressed versus nondepressed mother). If we are guided by the meta-analysis of depression and cortisol secretion, the authors should have obtained a group difference in recovery, which they didn’t. I really doubt this is reproducible in a larger, independent sample, with transparently reported statistics.

Recognize what we have here: prestigious journals like Molecular Psychiatry have a strong publication bias in requiring statistical significance. Authors therefore must chase and obtain statistical significance. There is miniscule difference from p<.042 and p<.06 – or p<.07, for that matter – particularly in the context of multivariate statistics being applied to skewed and winsorized data. The difference is well within the error of messy measurements. Yet if the authors had obtained p<.06 or p<.07, we probably wouldn’t get to read their story, at least in Molecular Psychiatry.*

Stay tuned for my next installment in which I compare results of this study to the press release and coverage in Insel’s personal blog.  I particularly welcome feedback before then.

*For a discussion of whether “The number of p-values in the psychology literaturethat barely meet the criterion for statistical significance (i.e., that fall just below .05) is unusually large,” see Masicampo and LaLande (2012)  and Lakens (2015).

Category: biomarkers, HPA Axis, hype, maternal depression, Mind-body, stress | Tagged , , , , | 2 Comments

Will following positive psychology advice make you happier and healthier?

smile or dieSmile or Die the European retitling of Barbara Ehrenreich’s realist, anti-positive-psychology book Bright Sided:How Positive Thinking Is Undermining Americacaptures the threat of some positive psychology marketers’ advice: if you do not buy what we sell, you will face serious consequences to your health.

Barbara Fredrickson, along with co-authors including Steven Cole, make the threat that if we simply pursue pleasure in our lives rather than meaning, there will be dire consequences for our immune system by way of the effects on genomic expression.

People who are happy but have little-to-no sense of meaning in their lives have the same gene expression patterns as people who are enduring chronic adversity.

A group consisting of Nick Brown, Doug McDonald, Manoj Samanta, Harris Friedman and myself obtained and reanalyzed the data on which Fredrickson et al based their claim. We concluded:

Not only is Fredrickson et al.’s article conceptually deficient, but more crucially statistical analyses are fatally flawed, to the point that their claimed results are in fact essentially meaningless.

objecrtive approach to moralIn workshops, books, and lucrative talks to corporate gatherings, Fredrickson promises that practicing the loving-kindness meditation that she markets will send you on an upward spiral of physical and mental health that ends who knows where.

My co-authors – this time, Nick Brown, Harris Friedman and James Heathers– and I examined her paper and obtained her data. Re-analyses found no evidence that loving-kindness meditation improved physical health. The proxy measure for physical health in this study – cardiac vagal tone – is not actually reliably related to objective measures of physical health and probably wouldn’t be accepted in other contexts. And it was not affected by loving-kindness meditation anyway.

Katrinaloverimages http://www.fanpop.com/clubs/katerinalover/images/30154750/title/dont-worry-happy-photo

Katrinaloverimages
http://www.fanpop.com/clubs/katerinalover/images/30154750/title/dont-worry-happy-photo

The simplest interpretation of Fredrickson’s interrelated and perhaps overlapping studies of loving-kindness meditation is that lots of people drop out from follow-up and any apparent effect of the meditation is actually due to unexplained deterioration in the control group. And though data concerning the participants’ practice of mediation were collected, none were presented concerning whether participants assigned to mediation actually practiced it or how it affected physical and mental health outcomes. Why were the data collected if they were not going to be reported? They could be used to address the crucial question of whether actually practicing meditation affects health and well-being.

Another queen of positive psychology advice, Sonia Lyubomirsky, proclaims in a highly cited paper:

The field of positive psychology is young, yet much has already been accomplished that practitioners can effectively integrate into their daily practices. As our metaanalysis confirms, positive psychology interventions can materially improve the wellbeing of many.

I showed these claims are based on a faulty meta-analysis of methodologically-poor studies. In addition to Lyubomirsky’s highly-cited meta-analysis, I examined a more recent and better meta-analysis by Bolier and colleagues. It showed that the smaller and poorer-quality a study of positive psychology interventions is, the stronger the effect size. With the more recent studies included in Bolier’s meta-analysis, I concluded:

The existing literature does not provide robust support for the efficacy of positive psychology interventions for depressive symptoms. The absence of evidence is not necessarily evidence of an absence of an effect. However, more definitive conclusions await better quality studies with adequate sample sizes and suitable control of possible risk of bias. Widespread dissemination of positive psychology interventions, particularly with glowing endorsements and strong claims of changing lives, is premature in the absence of evidence they are effective.

I’m quite confident that this conclusion holds for effects on positive affect and general well-being as well.

Actually, when Lyubomirsky attempted to demonstrate the efficacy she claims for positive psychology interventions, she obtained null results but relegated her findings to a book chapter that was not peer reviewed. Yet, her marketing of the claim that positive psychology interventions improve well-being continues undaunted and gets echoed in the most recent papers coming out of the positive psychology community, such as:

Robust evidence exists that positive psychology interventions are effective in enhancing well-being and ameliorating depression.

Advice gurus claim that practicing positive psychology interventions will lead to health and well-being without a good scientific basis. But another literature attempts to identify small changes in everyday and laboratory behavior that can have lasting benefits. These studies are not explicitly evaluating interventions, but the claim is that they identify small behaviors with potentially big implications for well-being and happiness.

Let’s start with an example from the Wall Street Journal (WSJ):

Walk this way: Acting happy can make it so

Research shows people can improve their mood with small changes in behavior

Elizabeth Dunn, Associate Professor of Psychology at the University of British Columbia, provides an orientation:

There are these little doses of social interactions that are available in our day” that can brighten our mood and create a sense of belonging. “I don’t think people recognize this.”

depressed postureThe article starts with a discussion of work by Johannes Michalak from the Department of Psychology and Psychotherapy at Witten Herdecke University, Germany. In one study, 30 depressed psychiatric inpatients were randomized to instructions to sit in either a slumped (n =15) or an upright (n =15) position and then completed a memory test. The idea is that an emotion like depression is embodied. Adopting a slumped posture should increase a depressive negative bias in recall. The abstract of the original article reports:

Upright-sitting patients showing unbiased recall of positive and negative words but slumped patients showing recall biased towards more negative words.

Michalak conducted another study in which the gait of 39 college students was manipulated with biofeedback so as to simulate either being depressed or nondepressed as they walked on a treadmill. During the period on the treadmill, the experimenter read 40 words to them and they were tested for recall. The abstract of the original study reports:

The difference between recalled positive and recalled negative words was much lower in participants who adopted a depressed walking style as compared to participants who walked as if they were happy.

As would be with expected with such small sample sizes, results were weak. Analyses were unnecessarily complicated. It’s not clear that effects would persist if more basic statistics were  presented. For instance, did patients assigned to the “depressive” slump condition in the first study recall fewer positive and more negative words, or both, or neither? Certainly in the second study with college students, there were no differences in recall of positive words and only small differences in recall of negative words. Claims in the abstract were based on the construction of a more complicated composite positive variable.

Michalak is following a familiar strategy in the positive psychology literature – indeed, one that is more widely followed in psychology: If you cannot obtain positive findings in straightforward, simple analyses, then (1) adopt flexible rules of analyses, such as selective introduction of covariates and making up new composite variables; (2) don’t report the simple statistics and analyses in tables where readers could check them; and (3) spin your results in the abstract because that is what most readers will rely on in deciding what your study found

Michalak claims that these studies point to manipulation of the embodiment of depression as a means of treating depression:

There is a mutual influence between mood and body and movement…There might be specific types of movements that are specific characteristics of depression and this feeds the lower mood. So it’s a vicious cycle.

Presumably, with this as a premise, depressed patients could obtain a clinically-significant improvement in mood if they sat up straight and walked faster. Maybe, but this is at best speculative and premature. Michalak does not directly test the take-away message the author of the WSJ article wants to give: Even if you are not depressed, you can improve you mood by sitting up straight and adopt what Michalak calls a “happy walking style.”

To be fair to Michalak, he may be pumping up the strength and significance of his findings and promoting himself a bit. But unlike the rest of the authors discussed in the WSJ article, he is not yet prematurely turning some scientific papers of modest significance and strength of findings into press releases, TED talks and positive psychology products like books and workshops.

But let’s turn to the work of Nicholas Epley that is next described in the article. Epley is a Professor of Behavioral Science, University of Chicago Booth School of Business and author of Mindwise: How We Understand What Others Think, Believe, Feel, and Want. Epley does not have a TED talk, but got mentioned in the Business Blog of the Financial Times as not needing one. And he’s available through the Washington Speakers Bureau, whose website proclaims it is “connecting you with the world’s greatest minds”.

According to the WSJ article:

“I used to sit in quiet solitude on the train,” Dr. Epley said. “I don’t anymore. I know now from our data that learning something interesting about the person sitting next to me would be more fun than pretty much anything else I’d be doing then,” he said.

This is a reference to his article

Epley, N., & Schroeder, J. (2014). Mistakenly seeking solitude. Journal of Experimental Psychology: General, 143(5), 1980.

talking to strangersThe study actually involves trains, buses, and taxis and giving different participants instructions to either connect with strangers, remain disconnected, or commute as normal.

In the train experiment, a composite measure was substituted for a simpler measure of whether these various strategies made participants happier:

To obtain an overall measure of positivity, we first calculated positive mood (happy minus sad), then standardized positive mood and pleasantness, and then averaged those two measures into a single index.

Epley study one

Error bars represent the standard error around the mean of each condition.

A one-way analysis of variance indicated a significant difference among the three conditions (p< .05) explaining a modest 6% of participants’ variation in the composite measure of mood. However, consulting Figure 1 in the article suggests the effect was in the difference between instructions to remain disconnected versus the other two conditions, not between connecting with strangers versus commuting as normal. See an excerpt of Figure 1 to the left.

Results were somewhat stronger when the experiment was replicated on buses, (p = .02), with 10% of the variance in participant mood explained by the condition to which participants were assigned. See figure to the right.Epley bus experiment 2

When the experiment involved talking to a taxi driver, significant results were obtained (p <.01). But this time, pairwise differences between conditions were tested and there was no significant difference between the connecting and the control condition, only between the control condition and the condition in which participants were instructed not to talk to the taxi driver.

This may not be rocket science, but it is apparently worthy of press releases, media coverage, and positive psychology products. The results are overall weak and may even disappear in straightforward analyses with simple measures of happiness. The most robust interpretation I could construct was if someone asks you to refrain from talking to others on the train or bus or even a taxi driver, you probably should ignore them. I offer this advice for free, and have no intention of presenting it in a TED talk with unattributed anecdotes.

Re-enter Elizabeth Dunn, Associate Professor of Psychology at University of British Columbia. Dr. Dunn is the author of Happy Money: The Science of Smarter Spending, has presented a  TED talk. She is available as a speaker through the Lavin Agency, which according to its website, is “making the world a smarter place.” The WSJ article reports on:

Sandstrom, G. M., & Dunn, E. W. (2013). Is efficiency overrated? Minimal social interactions lead to belonging and positive affect. Social Psychological and Personality Science,

Participants were instructed to either avoid any unnecessary conversation with a barista at Starbuck’s and simply be efficient in getting their coffee or:

“have a genuine interaction with the cashier—smile, make eye contact to establish a connection, and have a brief conversation.”

The journal article reports that participants instructed to make a “genuine connection” had more positive affect and less negative affect than those instructed to avoid unnecessary conversation. Unfortunately, unlike the Epley experiment, we are not given any comparison with a control condition, which would’ve clarified whether the effect was primarily due to instructing participants to have a “genuine connection” or to avoid conversation.

Then there are Professor Dunn’s student Jordi Quoidbach’s chocolate experiments, which have been promoted not only in this WSJ article, but in Dunn’s op-ed in the New York Times, “Don’t indulge. Be happy.”

The first of the studies was:

Quoidbach, J., Dunn, E. W., Petrides, K. V., & Mikolajczak, M. (2010). Money Giveth, Money Taketh Away: The Dual Effect of Wealth on Happiness. Psychological Science.

stack-50-euro-bills-isolated-white-19404122The study involved priming participants with a reminder of wealth – a photo of a large stack of Euro bills – or a similar photo that was blurred beyond recognition in the control condition. The 40 participants were then instructed to eat a piece of chocolate and complete a follow-up questionnaire.

As seen in the other studies, simple analyses were suppressed in favor of a more complex analysis. Namely, preliminary examination of the data revealed that female participants savored chocolate more than males. So, rather than simple t-test, analyses of covariance were conducted with gender and prior attitude towards chocolate as control variables. Note that they were only 20 participants per group to begin with, and so results of these multivariate analyses are quite dubious. Participants primed with the money photo spent less time eating the piece of chocolate and were rated by observers as enjoying it less.

Studies involving priming participants with seemingly irrelevant, but suggestive stimuli such as this one are now held in low regard and some feel they have contributed to the crisis of confidence in social psychology. Nobel Prize winner Daniel Kahneman suggests the lack of replicability of social priming research is “a train wreck looming” for social psychology. Results are often too good to be true and cannot be replicated. Jordi Quoidbach and her colleagues cite studies by Vohs(1,2) as supporting the validity of the manipulation, however, the two primary studies by Vohs could not be independently replicated.

Overall, this is an underpowered-study with results that probably depended on flexible analyses rather than simple ones. We would probably ignore it, except it appeared in the prestigious journal Psychological Science and has been hyped in the media and positive psychology products.

The second chocolate study was:

Quoidbach, J., & Dunn, E. W. (2013). Give It Up: A Strategy for Combating Hedonic Adaptation. Social Psychological and Personality Science, 4(5), 563-568.

chocolateThe study involved asking 64 participants to eat two pieces of chocolate into lab sessions separated by a week. Analyses were based on the 55 who showed up for the second session. Participants had been randomized to one of three conditions: restricted access (n = 16) in which participants were told not to buy any chocolate until the next lab session; abundant access (n = 18) in which participants were given two pounds of chocolate and told to eat as much as they comfortably could before the next lab session; and a control condition in which no explicit instructions (n = 21) were given. In the second lab session, all participants were given a second piece of chocolate.

Once again we have a small study in which the authors deny readers an opportunity to examine simple statistical tests of what should be a simple hypothesis: that restricted versus free access has an effect on enjoyment of a piece of chocolate. Instead of a simple one-way analysis of variance, the authors looked at their data and decided to do the (unnecessarily) more complex analysis of covariance. Nonetheless, we can still see that in pairwise comparisons between groups, there are differences between the restricted access and the abundant access and control group. Yet there were no differences between the abundant access instructions and having no instructions for what to do in the week between sessions.

The authors did not provide readers with appropriate analyses of group differences in changes in overall positive affect between the two sessions. Nonetheless, within-group t-tests revealed a decline in overall positive affect only for the abundance condition.

So, another small study in which positive results probably depended on tricky flexible analyses. We would not be discussing this if it were not in a relatively prestigious journal, discussed in the WSJ, and written about by one of the authors in an op-ed piece in the New York Times. I invite your comparison of my analysis to the hyped presentation and exaggerated significance for the study claimed in the op-ed.

This blog post ends quite differently than I originally intended. I wanted to take some highly-promoted findings in the positive psychology literature about the effects of small things on overall well-being. I looked to a WSJl article reporting findings from basically prestigious journals with recognizably big name promoters  of positive psychology.

I had expected that positive psychology people out promoting their work and selling their products could surely come up with some unambiguous findings. I could then discuss how we could decide whether to attempt to translate those findings into strategies in our everyday lives and whether we could expect them to be sustained with any lasting impact on our well-being. Unfortunately, I didn’t get that far. Findings turned out to be not particularly positive despite being presented as such. That became an interesting story in itself, even if I will still have to search for robust findings from the positive psychology literature in order to discuss the likelihood that following “scientific” positive psychology advice will make us happier overall.

Despite heavily-marketed claims to the contrary, positive psychology interventions do not consistently improve mental or physical health and well-being. The myth that these interventions are efficacious is perpetuated by a mutually-admiring, self-promotional collective that protects its claims from independent peer review and scrutiny.

As with the positive psychology intervention literature, it is a quick leap from the authors submitting a manuscript to a peer-reviewed journal to making claims in the media, including op-ed pieces in the New York Times, and then releasing products like workshops and books that are lavishly praised by other members of the positive psychology community.

It is apparently too much to expect that positive psychology advice givers will take time out from their self-promotion to replicate what are essentially pilot studies before hitting the road and writing op eds again. And too much to expect that the Association of Psychological Science journals Psychological Science and Social Psychological and Personality Science will insist on transparent reporting of adequately powered studies as a condition for publication.

The incentives for scientifically sound positive psychology advice just aren’t there.

Special thanks to the Skeptical Cat, who is smarter, more independent,  and less easily led than the Skeptical Dog.skeptical sleuth cat 8 30 14-1

 

Category: depression, happiness, positive psychology, self-help | Tagged , , , , , | 4 Comments