“A lie that is half-truth is the darkest of all lies.”
― Alfred Tennyson
Last week, a new study published in the Journal of the American Medical Association received a lot of media attention. The study, performed by Cynthia Ogden and colleagues at the CDC, aimed to describe the prevalence of obesity in the US and look at changes between 2003 and 2012. The study itself had several interesting findings, not least among them that the prevalence of obesity seems to have stabilized in many segments of the US population. However, they made one observation that caught the media’s attention:
“There was a significant decrease in obesity among 2- to 5-year-old children (from 13.9% to 8.4%; P = .03)”
This is where things get interesting, as the focus was not on the 5.5 percentage points difference. Instead of reporting the absolute difference, i.e. how much something changed, news outlets focused on the relative difference, i.e. how much they changed compared to each other. In that case, it would be (5.5/13.9 =) 40%. Which is much more impressive than the 5.5% change reported in the study. So you can guess what the headlines loudly proclaimed:
The media latched onto this “40%” statistic and ran with it, despite the researchers clearly stating that this was not their intention. In fact, from the paper itself, they said (in their conclusions):
Overall, there have been no significant changes in obesity prevalence in youth or adults between 2003-2004 and 2011-2012. Obesity prevalence remains high and thus it is important to continue surveillance. (emphasis mine)
This makes me wonder how many journalists read the article, how many got to the end, and how many just saw what other people had reported and ran with the same headline and narrative.
Here’s the thing – they’re technically correct (the best kind of correct). Yes, childhood obesity dropped 40% based on that report, and if that is true, that is a dramatic decrease. However, that is one group, and even the researchers themselves conclude this may be meaningless. It begs the question why, and if this is an actual association or just an artifact of something else like the type and number of statistical tests used. But since the narrative had already been written, everyone followed suit, and next thing you know we’re all slapping hi fives and proclaiming that there has been a drop off in childhood obesity that may not actually be something worth celebrating.
Now, had the results been portrayed fairly, two things would have happened. For one, the findings would not have been as positive as they are now. In fact, the headlines would have read “Business as usual: Obesity the same for the last decade” or “Obesity up 20% among elderly women!” (The latter refers to the finding that the prevalence of obesity went up among women aged 60 years and older from 31.5% to 38.1%). Secondly, a much more detailed discussion of the study findings would have happened – why has the prevalence stabilized? Have we finally reached saturation? Are all the people who could be obese now obese? Or is something else going on? But these weren’t the findings that were focused on.
The worst outcome of this media exposure won’t be felt right now. It will be felt in the next study. You see, this study in JAMA was reported all over the media, and millions would have heard about how we’ve finally “turned a corner in the childhood obesity epidemic” (to quote the New York Times). Unfortunately, this may not be the case, and if a new study comes out saying the opposite, this further undermines the public’s confidence in science, even though the researchers in question never made any such claim.
And that, dear readers, is the darkest of all lies.
Ogden, C. L., Carroll, M. D., Kit, B. K., & Flegal, K. M. (2014). Prevalence of Childhood and Adult Obesity in the United States, 2011-2012. JAMA, 311(8), 806-814.