Memories of famous studies do not serve us well
A journalist emailed me a request for an interview about the work of Ellen Langer. I was busy hopping from Leipzig to Istanbul and on to Groningen with undependable Internet quality. So I suggested that we communicate by email instead of Skype.
I recalled Langer’s classic study being portrayed as demonstrating that nursing home residents provided with a plant to take care of lived longer than if they had a plant tended by the staff. I had not reread the study recently, but was sure that that was an oversimplification of what was done in the study and its findings. Yet, I knew this was still the way the study was widely cited in the media and undergraduate psychology textbooks. I did not recall much doubt being expressed.
From my current perspective, we should always be skeptical about small psychosocial studies claiming effects on mortality. Especially when the studies were not actually planned with mortality as the primary outcome, or with appropriate participant selection, uniformity of baseline characteristics between groups, and experimental controls. These are needed to guarantee that the only differences between experimental and control conditions are attributable to by group assignment.
There is a long history of caims about mortality found in small studies not holding up under scrutiny. The consistency of this happening should provide the prior probabilities with which we approach the next such claim. Very often, apparent positive results can be explained by data have been manipulated to support such a conclusion Even when we cannot see precisely how the data were manipulated, apparent effects cannot be replicated.
Surely there may yet be a replicable finding lurking somewhere out there.
But I operate with the principle “Beware of apparent evidence of improved survival from small [underpowered] psychosocial studies not designed to look for that particular effect.” I begin examining a new claim with a healthy dose of skepticism. And we should always ask: by what plausible, established or testable biological mechanism would such an effect be expected?
Death is a biomedical outcome. I continue to be amazed at how people, even professionals, who would be dismissive of claims for medical interventions extending life be made post hoc from tiny studies nonetheless going gaga when the intervention is psychosocial. I’ve come to appreciate that finding a link between manipulating the mind and extending life is a holy grail that keeps getting pursued, despite a history of nothing being found.
I’ve debunked claims like that of Drs. David Spiegel and Fawzy Fawzy [sic] about psychological interventions extending the lives of cancer patients. These claims didn’t hold up under careful scrutiny. And findings claim for these original studies cannot be replicated in larger, better designed studies.
Why should we care? Isn’t keeping hope alive a good thing? Claims about the mind triumphing over body are potentially harmful not only because they mislead cancer patients about their abilities to control their health outcomes. More importantly, cancer patients are left with the mistaken belief that they succumbed to cancer, they and their loved ones can blame them for not exerting appropriate mind control.
In my email to the journalist, I expressed skepticism and was eventually quoted:
The study that arguably made Langer’s name — the plant study with nursing-home patients — wouldn’t have “much credibility today, nor would it meet the tightened standards of rigor,” says James Coyne, professor emeritus of psychology at the University of Pennsylvania medical school and a widely published bird dog of pseudoscience. (Though, as Coyne also acknowledges, “that is true of much of the work of the ’70s, including my own concerning depressed persons depressing others.”) Langer’s long-term contributions, Coyne says, “will be seen in terms of the thinking and experimenting they encouraged.”
The journalist had told me his article would appear in the New York Times at the end of October. At first, I didn’t bother to check, but then I saw the video of the CBS morning News extended interview with Langer. It was shocking.
I learned of her claims from an unpublished study that she could lower blood glucose levels of women with diabetes by manipulating their sense of time. And she had lowered women’s blood pressure by giving them a hair cut and coloring. And now she had wild plans to attempt to shrink the tumors of women with metastatic breast cancer.
The New York Times article had a title that warned of hype and hokum ahead –
The Times article described plaintiff’s intervention:
Langer gave houseplants to two groups of nursing-home residents. She told one group that they were responsible for keeping the plant alive and that they could also make choices about their schedules during the day. She told the other group that the staff would care for the plants, and they were not given any choice in their schedules. Eighteen months later, twice as many subjects in the plant-caring, decision-making group were still alive than in the control group.
To Langer, this was evidence that the biomedical model of the day — that the mind and the body are on separate tracks — was wrongheaded.
The study was conducted in the 70s, in the days of showman research like that of Phil Zimbardo (whom Langer had as a professor) and Stanley Milgram. Many of the social psychological studies of that period were more carefully attempts at dramatic demonstration of the investigators’ ideas, than rigorously controlled experiments. And they don’t hold up to scrutiny. You can find thorough debunking of the Zimbardo Stanford prison experiment – which had only 24 student guard and prisoners – as well as Milgram’s obedience study, but the perceptions of such work seem immune to being unsettled.
In re-examining Langer’s 70′s nursing home study, we shouldn’t chastise her for not anticipating later developments like CONSORT or doubts being expressed about strong post-hoc claims made from underpowered studies. But we should not now be uncritically accepting such flawed, older studies as evidence, particularly when it comes to physical health effects and mortality. So, I’m going to be looking at the validity of currently citing Ellen Langer’s work, not chastising her for what she didn’t do in the 70s.
You can find a PDF here of
Rodin, J., & Langer, E. J. (1977). Long-term effects of a control-relevant intervention with the institutionalized aged. Journal of Personality and Social Psychology, 35(12), 897.
The study is described as a follow-up of an earlier study, although the relationship between the two articles is more complex and of interest in itself. You can find a PDF here of
Rodin, J., & Langer, E. (1976). The effect of choice and enhanced personal responsibility for the aged: A field experiment in an institutional setting. Journal of Personality and Social Psychology, 34(2), 191-198.
The intervention was more complex than simply giving nursing home residents plans to tend.
A later independent attempt at replication describes Langer’s original intervention in succinct and accurate terms [but you can click here to get the full details in Langer’s article]:
In their study, an intervention designed to encourage aged nursing home residents to feel more control and responsibility for day-to-day events was used. One group of residents was exposed to a talk delivered by the hospital administrator emphasizing their responsibility for themselves. A second group heard a communication that stressed the staff’s responsibility for them as patients. These communications were bolstered by offering to subjects in the experimental group plants that they could tend, whereas residents in the comparison group were given plants that were watered by the staff.
Follow the numbers!: Scrutinizing how Langer got from the first study to the second
The original 1976 study reported
There were 8 males and 39 females in the responsibility-induced condition (all fourth-floor residents) and 9 males and 35 females in the comparison group (all second-floor residents). Residents who were either completely bedridden or judged by the nursing home staff to be completely noncommunicative (11 on the experimental floor and 9 on the comparison floor) were omitted from the sample. Also omitted was one woman on each floor, one 40 years old and the other 26 years old, due to their age. Thus, 91 ambulatory adults, ranging in age from 65 to 90, served as subjects.
However, statistical analyses reported in table 1 of the article were based on 24 residents being in the responsibility induced condition and 28 in the comparison condition. A footnote explained
All of the statistics for the self-report data and the interviewers’ ratings are based on 45 subjects (25 in the responsibility-induced group and 20 in the comparison group), since these were the only subjects available at the time of the interview.
The 1977 follow-up study reported
Twenty-six of the 52 were still in the nursing home and were retested. Twelve had died, and 14 had been transferred to other facilities or had been discharged The differences between treatment conditions in mortality are considered in a subsequent section The groups did not differ in transfer or discharge rate Only 9 other persons from the original sample of 91 were available for retesting Since they had incomplete nurses’ ratings in the first study, they are only included in follow-up analyses not involving change scores in nurses’ evaluations Almost all of the participants now lived in different rooms, since the facility had completed a more modern addition 13 months after the experimental treatment.
The 1977 follow-up study supplemented these data with a new control group of residents who had not participated in the original study.
We also evaluated a small control group of patients who had not participated in the first study due to a variety of scheduling problems Five had previously lived on the same floor as subjects in the responsibility-induced condition, and 4 lived on the same floor as the comparison group All were now living in the new wing. The average length of time in the nursing home was 3.9 years, which was not reliably different for the three groups.
The 1977 follow-up study reported 18 month follow-up data, with 7 deaths for the 47 (15%) of the residents in the responsibility-induced intervention group and 13 deaths (30%) in the composite comparison group. These data were subject to an arcsine transformation frequencies and described as statistically significant (z = 3.14, p< .01).
An erratum statement corrected the z-score to z = 1.74, p<.10.
The outcome is therefore only marginally significant, and a more cautious interpretation of the mortality findings than originally given is necessary.
The APA electronic bibliographic resource, PsyINFO does not indicate that the existence of the erratum when the study is accessed, nor is there an indication at the journal. According to Google scholar, Langer’s nursing home study has been cited 848 times, but the erratum cited only 6 times. Surveying accounts of this study in the scientific literature and social media, I see no indication that there were no significant differences. Sure, it was a small study, but we can’t assume that accumulation of more participants would preserve the 2:1 difference between groups. Highly unlikely.
Richard Shultz and Barbara Hartman Hanusa followed up on Shultz’s earlier study investigating effects of manipulations of control and predictability on the physical and psychological well-being of nursing home residents. Like Langer’s study, the examination of mortality was post-hoc.
The original Shultz study was a randomized field experiment involving four groups:
(1) A control-enhanced condition in which the nursing home residents could control the frequency and length of visits from college student volunteers.
(2) A yoked group of residents who got the same frequency and length of visits, but without having control.
(3) A predict condition in which the residents were told when the volunteers were would visit, but could not control these visits were not informed how long they would last.
(4) A no-visit control condition.
Mortality data were examined in the follow-up article:
Two persons in the predict group and one person in the control-enhanced group died prior to the 24-month follow-up. A fourth person, also in the control-enhanced group, died between the 30- and 42-month follow-up.4 Fisher’s exact probability test was used to analyze these data (Siegel, 1956). Combining the no-treatment with the random group and the predict with the control-enhanced group yields a marginally significant Fisher’s exact probability of .053.
Like Langer’s study, this one involved post-hoc construction of lumped and split off groups. The marginally significant results would be radically changed by addition or removal of a single participant.
The collective memory of Langer’s study
The Langer study continues to be hailed as a breakthrough study in both the thinking of Langer and the larger field in terms of the powers of predictability and control in mind-body interventions seeking effects on mortality.
Aside from not achieving significant effects, the study is deficient by contemporary standard in numerous ways. The mortality effect is not arise in a randomized trial, nor even the original sample, but involves a post-hoc construction of a comparison group. There’s a high risk of bias in terms of a lack of
- Allotment concealment.
- Blinding of investigators and probably nurse raters.
- Equalization of baseline characteristics in the two groups.
- Rigorous experimental control of what happened between randomization and final follow-up.
There were noteworthy changes in conditions in the nursing home in the interim.
The complex intervention is not reducible to simply giving nursing home residents responsibilities for plants.
The study nonetheless makes claims about a biomedical outcome, mortality. Even in 1978, the study would have been judged deficient if the intervention had been biomedical, rather than psychosocial. It certainly would not have gotten as much attention either then or now.
Ellen Langer’s incessant self-promotion has had no small part in preserving and extending the claims of mortality effect. Note that the study is widely referred to as the Langer study rather than the Rodin and Langer study.
Langer does not provide any plausible mechanism by which effects on mortality could have occurred. Overall, she asserts that the intervention manipulated responsibility and control, but even if that is the principal psychological effect, is on clear how this translates into better all cause mortality in a population already afflicted by diverse life-threatening conditions.
We have to be careful about the tooth fairy science involved in trying to test mechanisms for effects we don’t even know exist. But whatever occurred in this study is unlikely to be tied to the residents having to take care of plants. Yet that is one of features that is so enticing. All of us have anecdotes about older people being kept alive by having to care for their dog or cat. But even if there is some validity to these observations, it’s unlikely the beneficial effects of having a pet are tied to the power of attitude, rather than the modification of health-related behaviors.The original 1976 study ended with
The practical implications of this experimental demonstration are straightforward. Mechanisms can and should be established for changing situational factors that reduce real or perceived responsibility in the elderly. Furthermore, this study adds to the body of literature (Bengston, 1973; Butler, 1967; Leaf, 1973; Lieberman, 1965) suggesting that senility and diminished alertness are not an almost inevitable result of aging. In fact, it suggests that some of the negative consequences of aging may be retarded, reversed, or possibly prevented by returning to the aged the right to make decisions and a feeling of competence.
Not so straightforward, and these noble sentiments are not empirically advanced by the actual results of these studies, particularly the follow-up study. Yet I have no confidence that debunking will have any effect on how it is been mythologized. Some people have a need for such examples of mind triumphing over bodily limitations, even if the examples are not true.