SciWriteLabs #6: BMJ editor Fiona Godlee on the Wakefield-MMR fiasco, the problems with peer review, and research oversight boards

Bookmark and Share

Yesterday, the British Medical Journal called for “a parliamentary inquiry into research which claimed that the MMR vaccine causes autism and bowel disease, following extraordinary new disclosures about what it calls the ‘elaborate fraud’ behind the work of Andrew Wakefield.” The BMJ call to action was ironically prompted by new data provided by one of Wakefield’s defenders, as the journal explains in a press release:

In May 2010, Wakefield, formerly a researcher at the Royal Free medical school in Hampstead, north London, was struck off the medical register over a raft of charges, including dishonesty in research published in the Lancet in 1998. And last January, the BMJ concluded that his claims linking MMR vaccine with autism and bowel disease were “an elaborate fraud.”

Now, the journal publishes further revelations about the research, removing any remaining credibility to the claim that Wakefield and his co-authors had discovered a new inflammatory bowel disease associated with MMR. Experts studying unpublished raw data submitted to the BMJ with a view to exonerating Wakefield say it provides no evidence of such disease and that almost entirely normal findings were misreported in the Lancet paper.

Published in February 1998, the paper claimed that 8 of 12 children with brain problems seen at the Royal Free hospital developed autism within days of MMR, and that 11 of the 12 had colitis. The paper triggered a decade-long storm of public anxiety, plummeting levels of vaccination, and the re-emergence of measles as an endemic disease in Britain and elsewhere.

This new information does nothing to exonerate Wakefield of fraud but nor does it reflect well on his 12 authors, says Godlee. “It is impossible to reconcile [the new data] with what was published in the Lancet. The paper talks of enterocolitis and a new bowel disease involving a putative “unique disease process.” How could two consultant histopathologists have reported healthy biopsies and then put their names to such a text?”

The BMJ has been at the forefront of investigating the MMR scare, and earlier this year Dr Godlee wrote to University College London reporting six more papers involving Wakefield which have aroused concerns. She believes that a continuing failure to get to the bottom of the vaccine scandal raises serious questions about the prevailing culture of our academic institutions and attitudes to the integrity of their output. Given the extent of involvement of senior personnel at the highest level, only an independent inquiry will be credible, she says.

Earlier today, Godlee spoke with me about her publication’s latest charges, as well as a number of issues relating to science writing and journalism. This is a lightly edited transcript of that chat.

SM: I want to start out with a question that has come up a lot in science writing circles over the past several months: What is the proper role for journalists when it comes to checking/vetting scientific work? I’ve spoken (and corresponded) with a number of scientists who have essentially said that peer review is the proper way to ensure new research is accurate and worthy of getting attention, and reporters should stick to informing the public about the results of those studies. The MMR-autism fiasco seems to put the lie to that notion: Here, Wakefield’s paper passed peer review and it was only when an investigative reporter named Brian Deer took up the case that the house of cards began to fall apart.

FG: Lot of important issues here. Did the peer review process fail in allowing Wakefield’s paper into the medical literature? Why does post-publication peer review tends to focus on scientific error rather than countenancing the prospect of fraud? What was the role of the mainstream media in giving Wakefield such a sympathetic hearing? All contributed to where we are now — not to mention the failure of the co-authors to fulfill their responsibilities.

SM: So do you think the peer review process failed in this case, or does the fact that the Wakefield paper was published in the first place highlight the intrinsic deficiencies of a system that’s designed to check data –which, as you point out, presumes that data is legit in the first place?

FG: Peer reviewers can only go on what they are sent – and what the Lancet‘s reviewers were sent was a falsified version of the children’s data, which they had no way of knowing. However, when peer review has succeeded in detecting fraud in the past, it has tended to be because the research looked “too good to be true.” I think the reviewers might have spotted this in the Wakefield case if they had been on the ball – the children almost all reported as having similar bowel pathology despite being (reportedly) a consecutive series; the high proportion of families (8 of 11) reportedly blaming the vaccine.

But hindsight is a wonderful thing. More open to criticism is the fact that the study itself was so poorly designed — relying on parental recall etc — especially considering that this is such an important topic for public health. It is this that should have kept it out of a high profile journal.

SM:  I’d agree with you about the fact that the basic science was so shoddy in Wakefield’s study that it likely didn’t deserve to be published in the first place. The point I’ve made in the past is that the media should have known that the conclusions Wakefield was making in his press conference announcing his result wouldn’t be warranted by a 12-child case study even if the data had all checked out.

Once the Lancet piece was published, what could/should the mainstream media covering the story have done differently? Was it inevitable that once this story was out there it was going to take off the way that it did?

FG: I agree. And had the media ignored the study as they should have done, we might all have just carried on our merry way and Brian Deer would never have been put onto the case by his editor and the fraud would never have been uncovered. Essay question: does scientific fraud that has no impact on medicine, health care, science matter?

In terms of the media’s response – initially in the UK it was was quite sober, focussing on the lack of evidence of a link. For reasons I haven’t properly looked into, but Ben Goldacre will be able to give you more on, in about 2002 the mood changed. The UK’s Daily Mail has a journalist named Melanie Phillips, who took up Wakefield’s case, as did the satirical magazine Private Eye, publishing a special “expose” on vaccine safety and championing Wakefield. The BMJ pubslihed an article that talked about the problem of the media taking a “balanced” approach in such stories — setting Wakefield and the scientific consensus against each other as if there was equal weight on both sides. This is traditional legalistic approach but doesn’t work well for some stories in the media. Climate change is an example.

SM: I couldn’t agree with you more about the problems with two-handed reporting of one-handed issues — I wrote a lot about that in my book about the vaccine panics of the past decades. I also spent a fair amount of time looking into the initial 1998 coverage, and I’m probably less forgiving of those initial reports than you are. It was a situation where you had Wakefield making outrageous claims that virtually everyone else in the field said weren’t supported by his research — I’d argue that’s a case where the correct decision isn’t to put the story out there and essentially let the reader decide, it’s to not publish the story in the first place. But that’s a whole other issue…

One of the charges leveled against Lancet editor Richard Horton back in 1998 was that he published Wakefield’s piece despite some pre-publication misgivings that had been made because he wanted to stir up some attention for his publication. For people not involved in academic publishing, I think that might have seemed surprising. With this latest series, the BMJ has now been publishing new, fairly explosive charges about Wakefield paper for close to a year. In January, Paul Offit said that in some ways, Brian’s initial BMJ series was beside the point: Wakefield’s theories were already known to be incorrect. That’s a view I’m sympathetic to. At this point, it seems as if every new revelation or investigation only serves to bring the story back in to the public’s consciousness. Did you ever wonder whether this particular story has already gotten all of the attention it deserves?

FG: Yes, I’m very sympathetic to that view. However, I am in no doubt that the three articles we published in January did break new ground. Since they were published I have been invited to speak to a range of different groups around the world — clinicians, academics, research scientists — and I’ve been surprised by the extent to which what we published was, in their view, truely new and important. No longer was the resesrch in the Lancet, which had impacted so much of their professional activities, just poor science –  it was actually falsified.

I also thought it was important that we shed light on the failures at the institution and the journal to act properly when allegations of misconduct were [first] raised [by Brian Deer] in 2004. All too often such allegations are swept under the carpet, and that’s what happened in this case. But once we had published the series of articles and editorials in January and the BMJ had called for UCL to establish an independent enquiry, I thought that would be us done.

Then we were sent by Wakefield, via an intermediary, the original pathology grading sheets, and we had to decide what to do with them.

SM: I’m not disputing that they broke new ground at all — that’s clearly true. The pathology reports are interesting. In your editorial, you are quite forceful about what they show. Yesterday, Nature published a news story that used those same reports (supplied by David Lewis) and even quoted Ingvar Bjarnason, who wrote one of the commentaries in BMJ — but in Nature, he seems to be saying something quite different:

But [Bjarnason] says that the forms don’t clearly support charges that Wakefield deliberately misinterpreted the records. ‘The data are subjective. It’s different to say it’s deliberate falsification,’ he says.

I’m not sure if I have a specific question about that — it’s just interesting to me how differently the same information can be explained/described.

FG: I think we have been quite careful about what we said in relation to the children’s pathology. The conclusion that the data in the paper were fabricated is based on a whole raft of inconsistencies which together mean that, in no single child can the Lancet paper be reconciled with the health care records. This was also true for the bowel pathology reports. In January, we knew there were clear discrepancies between the routine hospital pathology reports — done by senior pathologist and co-author Susan Davies — and what was reported in the Lancet. We had no information that could shed light on how and why these changes had been made. A “research review” by fellow senior pathologist and co-author Anwar Dhillon was alleged to have found that the children had enterocolitis.

What we now know, as a result of the surfacing of the grading sheets, is that Anwar Dhillon found nothing of the sort. Almost all the children’s biopsies were normal or showing what amounts to physiological change. I agree with Bjarnason that we cannot know about intent. What we know is that the Lancet report is very far from an accurate report of the children’s biopsies. And it is not tenable that Wakefield simply transcribed Dhillon’s tick box that said “non-specific” for some of the biopsies and from this built the edifice that includes enterocolitis and a unique disease process for 11 out of the 12 children.

SM: Before we need to go, I want to get to some questions sent in by readers. Here’s one:

“Does research oversight, which the BMJ has called for, really work?  The US version has pretty mild sanctions (they list the people sanctioned).  Also, what about when people like those who put out the Pace University study go around the rules? There isn’t anyone to sanction them.  They don’t depend on grants, so it isn’t like they will lose their government funding.  No one wants to pursue that.  HHS, NYU, Pace — they just act like nothing happened.”

FG: In regards to research oversight – that’s a real challenge, I agree. Although the US system may be weak, at least there is one. In the UK we have nothing and, as I have been increasingly saying when asked about this, we should be ashamed about this. The Wakefield example is perhaps the most florid, but no doubt there are others — the evidence fronm the Committee of Publication Ethics suggests this — where institutions are failing to properly investigate allegations. We need at least some system that places the responsibility on institutions to investigate and publish their findings

SM: Two related questions that came in over the transom:
* Given the large number of examples of ethical lapses and dishonesty proved against Andrew Wakefield, how does one communicate to the public without it becoming a great deal of noise?
* Isn’t the Wakefield story over?  This last attempt to exonerate him was pretty weak.  Are we supposed to respond to every “newly discovered” document he pulls out of a drawer?

FG: On communication with the public — I agree, it’s a big challenge, especially in the age of the Internet. The medical and scientific establishment can’t possibly compete with the public, and parents of children with autism and people with autism are understandably passionate about this issue. I’m not sure what the answer is, but openness and humility will go a long way.

On whether this should all be done and dusted by now — if only that were true.

SM: I do want to make sure I ask one more question before we need to go. Has the BMJ looked into (or does it have any plans to look into) a 2002 paper that appeared in the BMJ publication Molecular Pathology titled “Potential viral pathogenic mechanism for new variant inflammatory bowel disease“? Wakefield and John Walker-Smith were both authors on that paper. It’s a study, since fairly convincingly discredited, that used PCR data to “prove” the measles virus was in children’s guts.

FG:
We have referred his to UCL, the institution where the research took place. This is the way journals are supposed to deal with potential misconduct. Once we hear back from them, we will take whatever action is appropriate

SM: Isn’t that kind of passing the buck, especially if you think the UCL has failed in other instances regarding Wakefield’s work? The fact that there’s a review that’s theoretically supposed to occur doesn’t mean that publications can’t also conduct their own reviews…right?

FG: Journals don’t have the resources or the aegis to investigate authors for misconduct. Arguably that’s where the Lancet went wrong in 2004. It should have stepped back (instead of almost running the investigation itself) and if the institution [where Wakefield had conducted his research] failed to do a proper job, it should have then kept badgering them until they did so. This is what we are now doing in relation to Wakefield’s other work.

Related Posts Plugin for WordPress, Blogger...

Creative Commons License
SciWriteLabs #6: BMJ editor Fiona Godlee on the Wakefield-MMR fiasco, the problems with peer review, and research oversight boards by PLOS Blogs Network, unless otherwise expressly stated, is licensed under a Creative Commons Attribution 4.0 International License.

This entry was posted in Media, Science, SciWriteLabs, Vaccine safety and tagged , , , , , , , , , , , , , , , , , , , , , . Bookmark the permalink.

Comments are closed.