Today’s sessions at the “boot camp” were gripping in so many ways. First, I have to recommend looking up any and all research done by Steve Woloshin and Lisa Schwartz. This dynamic duo are co-directors of the Center for Medicine and the Media, at The Dartmouth Institute for Health Policy and Clinical Practice, and co-directors of the VA Outcomes Group, VAMC, in White River Junction, VT. They are wonderful speakers—plain-spoken, fact-based, and charismatic. It doesn’t hurt that they’re also very funny. I wrote here several months ago about Overdiagnosed, a book that they co-authored with Dr. Gil Welch, and today’s sessions with them were along the same lines as that book – facts and insights that are revelatory because of how different they are to the messages we normally here and to the way of thinking that we’ve allowed ourselves to be trained into in life.
One of the key points made today is the importance of framing, something that advertisers know and study, but which we regular folks might not think about too much. Or at least not this regular folk. So as an example was the Evista ad that claimed a 68% reduction in vertebral fractures in one year versus a placebo. But 68% of what? In who? When you parse the data of the study from which that 68% figure was derived, it turns out that the over 1 year, the risk of vertebral fracture with Evista was one third that seen with placebo. But here’s the thing: the actual risk percentages were .83% versus .27%. Those small numbers strike quite a different chord than the large-font, bold-faced 68%. Sometimes data can be visually framed in a way that causes distortion. An effect can look very dramatic if you scale it in a way that causes it to look dramatic, like having a Y axis that goes from 0 to 1, where small differences will end up looking very dramatic.
They also gave a wonderful analogy for understanding terms like “absolute risk,” “relative risk” and the like. The analogy is with shopping: Absolute risk can be thought of as the regular price of something. That’s the control group. Absolute risk is the sales price – that is what we use to understand the effect in the intervention group. Relative risk, which is what we’re talking about when we say that something has an X times higher risk than something else, is a fraction where the sales price is the numerator and the regular price is the denominator. Relative risk reduction can be thought of as how much of a percentage off is the sales price from the regular price. And finally, the absolute risk reduction can be thought of as the savings, or in other words, the regular price minus the sales price. I love this analogy and how it allows us to understand data presented in an abstract or full study in a way that, well, maybe doesn’t make our brains hurt so much, and definitely clarifies exactly what is being said and what really happened in the study (as you can probably guess, what *actually* happened in a study is often far more minimal than any subsequent presentation of the data, whether in an ad, a press release, a scientific meeting, or newspaper article, might lead us to believe).
There was much, much, much more, and all I can say is if you ever have a chance to attend a course with Steve Woloshin and Lisa Schwartz, run, don’t walk. In the meantime, here’s a link to their latest book.
Equally vital was our session with Sidney Wolfe, Director of the Public Citizen’s Health Research Group. This session was a real Matrix moment for me, filled with information that crumbled some ways I was thinking about drug development and the pharmaceutical industry. I was really holding out hope that it wasn’t as bad as many people thought, and now, well, I have to reexamine my thinking, particularly in light of the many examples Dr. Wolfe gave today about how slow the FDA has been about recalling unsafe drugs. Maybe I can post more about that another day, but for now I wanted to make sure to give a link to HRG’s website, www.citizen.org/hrg.I really recommend checking it out when you have the time.