Trine Tsouderos on This Week in Virology: When do you fact-check article content with sources?

I have long been a fan of Columbia University virologist, Dr. Vincent Racaniello. Vincent is an outstanding scientist who has also worked extensively make virology accessible to students with his own textbook and the award-winning Virology blog. For me, one of the best offshoot features of the blog has been his weekly podcast with Dick Despommier, This Week in Virology (TWiV).

This past week, TWiV Episode 149, spent the first 35 minutes or so interviewing Chicago Tribune science and medical writer, Trine Tsouderos, while at the International Conference on Antimicrobial Agents and Chemotherapy (ICAAC) in the Windy City. Because the netcast was held at the meeting, Vincent and the gang put together a very nice video of their panel discussion:

I’ve admired the writing of Trine Tsouderos for the last few years (you’ll learn that she is Norwegian-Greek and that her name is pronounced TREE-na soo-DARE-ose), particularly on the anti-vaccination movement and the questionable association of XMRV with chronic fatigue syndrome. I also learned some exciting homeboy things about her from the interview such as her undergraduate education at UNC-Chapel Hill in international studies, her taking organic chemistry for fun while she struggled with wanting to be a writer vs. go to medical school, and her first job covering minority and rural infectious disease issues in Wilson, North Carolina.

But what got a Twitter discussion going last night and this morning was Trine’s discussing her practice of sometimes running quotes, paragraphs, and even full articles past scientists she’s interviewed for fact-checking purposes. Particularly in cases where she is interviewing someone about complex original research literature, she expressed her motivation as the desire to get it right “because you can’t retract 300,000 newspapers.” (Trine, please correct me if I misrepresented what you said.)

The relevant section begins around 12:55 of the interview and runs through about 16:30 (but please do listen to the whole episode when you get a chance.).

She acknowledged that this is often a no-no in science reporting but I’ve now heard on Twitter from revered science journalists like John Rennie, Scott Hensley, and Maryn McKenna that some special cases may warrant such fact-checking and is often a sign of thoroughness. But Scott Hensley issued the cautionary tweet that, “Checking facts is 1 thing. Deputizing source as editor is another.”

Well, the response to this little Twitter banter leads me to think it may be valuable to bring the discussion out to more than 140-character bursts. For example, I know that Maggie Koerth-Baker, Science Editor at BoingBoing, has proposed a ScienceOnline2012 session on the problem of journalists growing too close to their sources as stimulated by her reading of the book, Wrong.

So, my dear professional science and health journalist friends: how do you negotiate this slope of prospectively sharing article content with scientist sources?

Please feel free to use more than 140 characters in your comments below.

Related Posts Plugin for WordPress, Blogger...
This entry was posted in Journalists, Awesome, Science Journalism, Stuff I Don't Know About. Bookmark the permalink.

128 Responses to Trine Tsouderos on This Week in Virology: When do you fact-check article content with sources?

  1. The few times I’ve done this (two, maybe?), I’ve done it for fact checking purposes only, i.e., for complex or nuanced information that requires review. I also always note that I will not alter quotes or anything unrelated to correcting the facts. Source does not have veto power over anything, only factual clarification powers.

  2. Dirk Hanson says:

    Hensley on the crucial distinction: “Checking facts is 1 thing. Deputizing source as editor is another.”

  3. John Rennie says:

    Thanks for opening the discussion, Dave.

    To recap some of what I mentioned to you on Twitter, I and a number of writers at Scientific American over the years sometimes showed portions of news stories or reported features to scientist sources while we were writing them. I don’t think I ever sent a story in its entirety, but I would sometimes send passages when I was concerned that some subtlety of the wording in scientific explanations might be wrong. It was in the interest of fact checking, and as such, I didn’t regard it as too different from saying to a source during an interview, “Let me see if I understand this right: you’re saying that….”

    If I did it—and it wasn’t my routine practice—I was always sensitive to whatever issues of newsworthiness might be involved to minimize the chances of undermining the story. Also, when I sent something to a source, it was always with an expressed understanding that it was purely for fact checking, and that I would not be bound to make any changes that the source wanted (or demanded). And when evaluating whatever suggested changes came back, I tried to be mindful of spin that the source might be trying to introduce. In the end, as the writer, the responsibility for what is right or wrong in the story has to rest with me.

    I completely agree with Scott Hensley that sharing stories in progress with sources is journalistically perilous, so trying to minimize or eliminate the need is always good. I’m also reluctant to engage in any special pleading to the effect that science journalism is somehow different from other journalism and so special rules should apply. Nevertheless, the danger of getting a story very significantly wrong through misunderstanding technical details is greater in science journalism than in many other types, so we need to do whatever we reasonably can to prevent errors from getting into print.

  4. Hi David, just want to say thanks so much for the props. I think, for anyone considering this sort of process, that it’s very important to make sure your sources know what you are expecting – and not expecting – when you do this. I try to make sure they understand I’m not looking for editing help and so on. I just want to make sure I quoted them correctly – in context – and that my facts are correct. It helps eliminate mistakes and surprises, without turning sources into editors, which would be a mistake. I don’t see many downsides, honestly.

  5. What she said, plus.
    One risk at some outlets is dismissal. A publisher can claim it is the sole determinant of what information its writers release and when. In such cases, I’ve verified lists of “facts” pulled from the story.
    The other risk is that the source will confuse verification and clarification with approval. On that point, the writer simply needs to hang tough.
    I suspect that some writers, despite a reputation for talking truth to power, get a bit weak in the knees on this one.

  6. Hi Eric,

    Indeed, although my take on it is that when I contact sources for fact-checking, I am doing it as a courtesy, as an extra step, and that I am not obligated to change anything at all. I often report out any suggestions, to ensure I am not being spun, or exchanging a correct statement for a false one.

    In my experience, while this is an extra step, it’s not extraordinarily time consuming. Yes, it’s an extra round of calls or emails, but my experience is that usually everything is just fine, there may be a few things I need to check again, and that’s it. And I can sleep well knowing I did everything I could possibly do to ensure the story is right.

  7. Mary Knudson says:

    Good discussion, David. Many, many years ago Boyce Rensberger, then science editor of the Washington Post, urged fact checking of parts or even all of a science story with sources. I don’t now remember if he thought the sources you fact checked with should be the sources you quoted or if he thought that you could show a passage you wanted to be sure you understood to a knowledgeable scientist who was not the one quoted in the story. Science writers responded pro and con to Boyce’s advice.

    I think the controversy stems from journalism’s roots in covering government and politics where there is much less reason to show quotes to politicians and much more unlikely. But in medical and science reporting, I think the writer’s (journalist or non journalist) job is first of all to get it right and this can be difficult, even for seasoned medical and science writers. I have no problem with showing a portion of a story to a source if I have any doubts about its accuracy. And I have never had the doctor or scientist source do anything more than respond on the accuracy. No source has ever tried to run editorial interference. I think it is important as Trine mentioned, to be clear when you contact the source that all you want is to verify that the information is correct.

    I also think that in the matter of a long story that is very involved and discusses a medical or scientific matter of controversy, it is fine to send the piece to a couple of sources a reporter feels are well qualified in the field (either someone quoted or not quoted in the article) to not only ask if the concepts written about are accurate but to get their view if overall the piece has mentioned the important points about an issue that is in the public interest. You may get called to your attention a significant study you had overlooked.

    The important matter is that you, the writer, are in the driver’s seat and you decide what suggestions or new information you wish to use or not use. Sources should never have editorial control over a story.

    Mary

  8. Grant says:

    Dipping into my copy of Elise Hancock’s ideas into words, I see she recommends fact checking via showing copy where it’s appropriate (see p. 46). I’ll try come back to this when I’ve more time, with any luck tonight. (I confess I’m happy reading here and finding that others with experience in science writing have similar views to my own naïve rumination.)

  9. Ed Yong says:

    The downside of doing this is that some people start asking for wholesale changes, tonal changes, or start going back on what they actually said (and meant).

    The upside, and I think this is significant, is that while journalists can fact-check specific things, we don’t always know the ways in which we can screw up. Unknown unknowns, and all that. An innocuous choice of word can make a sentence completely wrong and it can take an expert’s eye to spot that. No amount of “Let me run this one stat past you” will pick that up.

    I propose a simple but efficient way to get the upsides but not the downsides: run the piece/paragraph/whatever past an independent party who is not involved or quoted in the article, specifically for the purpose of spotting any nuances that you might have missed. This brings its own challenges – you have to pick someone you trust and who knows their stuff but doesn’t have a horse in the race. But those of us who are plugged into social networks should know plenty of people like that.

  10. John Rennie says:

    Sources can ask for any changes they like. I’ll only make ones that I think are right for the story. If writers aren’t prepared to stare down their sources over that principle, they shouldn’t be sending sources anything.

    Sending text to independent parties is mostly a good solution, but often the parts of stories I wanted to check had aspects of experimental details (for example) that only those involved could verify.

  11. Brendan Maher says:

    As an editor, my usual suggestion to writers who ask about this is to do fact checking over the phone. This avoids the sometimes inevitable loss of context that occurs when sending someone snippets of an article or straight quotes. The other, more time consuming practice is to distill down the parts of the story in which the source is mentioned without copying directly from the story and including the bits of context that are often important. E.G. “In a section of the story that deals with X, I have you saying something to the effect of Y”

    Still, as we know, much tone and intent can be lost via email.

    Phone calls are better for straightening out details and for some sensitive stories I have essentially read verbatim text and sometimes nearly full articles to sources to check more nuanced issues.

    I’m a strong believer in fact checking and although the problems from sending verbatim text aren’t common, I have struggled with them on both sides of the desk – as a writer and an editor. Doing fact checking as a conversation about the story tends to – in my experience – produce much better copy and a more nuanced understanding of the material in the end.

  12. David Kroll says:

    These are terrific insights – many thanks to all of you for contributing and special thanks to Trine herself for engaging with us here.

    In my own experience as a scientist being interviewed, my interactions with the online print side of television networks have almost exclusively been that I never see the copy in advance. But email interviews with one of these orgs (ABC News Medical Unit) has been great because they’ve used my unadulterated quotes and very accurately used my other text as background.

    The large national papers and even my then-hometown Denver Post never sent copy for fact-checking. But trade and professional publications have at least sent the section where I’ve been quoted for review for accuracy.

    I haven’t yet been graced with in interview by any of you fine folks but the science of my small laboratory has not yet quite gotten to the level of your writing.

    Keep the comments coming – this is very enlightening.

  13. DeLene says:

    This is a wonderful discussion, David, thank you so much for bringing it up. I think there are many benefits to this practice, if it is done well and if expectations are clearly defined by the writer in advance. As others have pointed out, policies of the editors or outlets that a writer is working for also come into play and must be adhered to.

    In my experience, as long as the writer defines for the source, in advance, what they are seeking in a pre-publication review, there are few problems. The beautiful thing about working with scientists is that they have the capability to think categorically. If you ask them to only check factual accuracy, or to check the appropriate use of illustrative statistics picked from their paper — then 99 percent of the time, that is what you will get back.

    I often send a paragraph or a passage I may be concerned about to a source for review. I’ve not once had a problem of someone going back on what they said, or asking for changes outside of what I asked them to check over. (I make it clear I won’t alter quotes.) Then again, I tend to not write about very controversial issues. Thinking back on when I’ve done this, nine times out of 10 it’s related to explanatory sections. It’s easy to truncate ideas, or leave out an important emphasis when doing this, and most times I just want to know that my two- or five-sentence way of describing something is still accurate even though it took the source 10 minutes to explain it.

    I’m using this practice for the 80,000 word book I’m writing on red wolves. The amount of material is so massive (175 cited sources so far, and more than three dozen interviews — many multiple times with the same source) that the sheer size of the info makes it more likely I could inadvertently introduce an error. While the univ. press that will publish my book sends their manuscripts out for external review before publishing, I’m in the midst of my own set of reviews before I even send it to them. I’m sending sections to a widow of a man whose field notes I used, to review for historical accuracy where I had to make inferences based on his patchy records; I’m sending sections that deal with origins to experts for review; and I’m sending sections where I wrote about field work to the biologists themselves. I was very clear, in each instance, of what I wanted them to check for. Dates, place names, numbers, etc. were to be verified, but I would not change literary issues, impressions/perspective, descriptions and such. So far, it’s been wonderful, and several errors have been caught. I feel strongly that this method, while time consuming and sometimes brutal to my fragile writerly ego, is making the book stronger.

  14. Aurora says:

    I like Ed’s suggestion of running things by a savvy independent party. I guess there’s a (slim) chance that something very specific to the paper you’re writing about might slip through, though, if it’s an unpublished paper that your independent ‘checker’ can’t access yet?
    I’m also wondering if/how press officers at research institutions (like myself) could help here. I usually do some ‘expectation management’ with scientists anyway, and do my best to ‘train’ them to fact-check, not edit, in my own interactions with them (writing up their research for various internal outlets) – but can I do more?
    Although I’m not a specialist, if I’ve written about that scientist’s work I’ll likely be attuned to some of the possible word-traps and nuances involved. If I’m included in the discussion (e.g. if I’m cc’ed in the request to the scientist), I can try to help. The question, ultimately, is: would you trust me to?
    As a little bit of an aside, I think another issue to consider when trying to make sure no nuances have been missed is language. Many scientists are not native-speakers of English, and although many have astoundingly good English, they’re still not always aware of some double-meanings, or can sometimes wrongly ‘transfer’ expressions from their mother-tongue, or unknowingly misuse English expressions (I once had a scientist who wanted to use ‘out of the blue’ to say something had first happened in the sea, for instance). Just something to keep in mind…

  15. Deborah Blum says:

    I’ve often had scientists ask me why more journalists don’t fact check their stories to avoid errors. Especially when science writers do. I point out that we’re a distinct subset here – does anyone expect, for instance, political reporters to check their quotes or accuracy of description with members of Congress? And think about how that would color our feelings about political coverage.

    So I don’t think anyone embraces this practice as a general professional rule. It’s specific to science/environmental/health journalism because of the complex and technical nature of the subject. And as a science writer I’ve been checking back on accuracy issues for years. I’m not a condensed matter physicist or a biochemist, I’m aware that no matter how much background research I do for a story, I still may miss nuances and small details. And the nuances and small details make the difference in this kind of journalism so I’ve always appreciated the generosity in time and trouble that many scientists have taken in helping me get it right.

    So many good points have already been raised in this discussion already. I also like phone checks rather than sending the entire text, partly because as mentioned, you run the risk of letting a source be your editor. I have occasionally gone through a whole story with a scientist though. This especially if it is a controversial subject. When I was at The Sacramento Bee, I spent a year working on a series about nuclear weapons design. One story was about the fact that the U.S. government was being dishonest about the state of the nuclear stockpile. That story was based on information from a cadre of scientists and I had them read it over to make sure that every detail was right. And I will tell you that as part of that same series, I interviewed the physicist Edward Teller and he refused to do the interview unless he could see the story in advance. I agreed (and that’s a definite judgment call). He changed almost nothing except a few technical details but he hated the story so much that he never spoke to me again, except once when I hunted him down at a press conference so that he would be forced to answer my question.

  16. Pingback: When do you fact-check article content with sources? Discuss. | digitalgrip.fieldnotes

  17. John Rennie says:

    Because so many of us have already weighed in favor of checking parts of stories with scientist sources, let me complicate matters by raising a hypothetical (?) objection.

    Suppose I’m working on a story about a new study that claims, say, to disprove global warming. I interview the principal investigator, write up the story, and email portions describing technical details to him for fact checking. He points out that I had a number of details wrong, and I work his corrections into my story. My final story, however, makes a firm argument that the paper is probably wrong. The aggrieved researcher decides to counterattack in his blog, and he uses my original, incorrect text as “proof” that I didn’t really understand the science involved and had been biased from the start. The credibility of my story consequently suffers.

    It could work the other way, too: Let’s say I wrote a story that favored this highly controversial paper. The researcher, pleased to find a reporter who seemed to take his side, blogs about our good interaction and even mentions that he helped me polish up some of the text. Those on the other side of the controversy might then point to this “collaboration” as evidence that I had been biased in his favor and failed to look at the work objectively. Again, my story’s credibility suffers.

    I’ve never known anything like these examples to occur, but given both how contentious some science topics can be these days and how much more publicly engaged many scientists are, I have to think they are less far-fetched than they once were. Or perhaps they demonstrate the superiority of Brendan’s suggestion above that phone calls are better than emails for fact checking—if only because there’s no paper trail! But if nothing else, I think they should underscore how careful reporters should be in choosing to show their stories to sources.

  18. John’s point is interesting, and a good one. Many of my stories wind up being controversial (my topics have been vaccines, autism, Lyme and CFS, among others), and at various times people have posted about their interactions with me during a story, so I always try to remember anything I am writing in an email may become public someday.

    All that said, I think the problem is solved by making sure the process is more or less the same for everybody in the story. I contact everybody for fact-checking, including sources I suspect will be unhappy with it. Everybody gets a call or email (sometimes many,, sometimes both) from me. That’s good for the story and it’s fair, and it ensures there will be no surprises the next day, and that the story is airtight. Or as airtight as I could possible make it.

    Anyway, that’s my process. I’m not convinced it is better than any other, but it has worked for me so far without many problems along the way. I don’t have a problem saying no to folks wanting to go back in time and change their quotes, I don’t have a problem politely ignoring their suggested edits, rewrites, etc. I don’t have a problem considering new information if they wish to send it, or doing a little more reporting if need be. I think it all helps the story, and makes it clear to my sources that my aim was always to get it right, even if they don’t like the story in the end.

  19. John, I guess doing the checking via phone rather than email might resolve the issues arising from that hypothetical, as it would leave nothing in bytes or bits that could be distributed.

  20. Maryn says:

    I was at a time-consuming conference and then traveling yesterday, so am late in responding. Of all the excellent comments above, John, Deborah and Brendan particularly speak for me.

    This will probably be a long comment, so I will say up front that I do not think it is a good idea to share the entire text of a newspaper or magazine story with the sources who are in it — because I have done it, and had it backfire, as I’ll explain.

    Now, at more length: I do believe in factchecking. I have an MS and half of an MPH and I never assume that I understand what I am writing about better than my source (though, because I am a deep-driller in research, I do sometimes find I know the literature better than they do). I have a variety of practices, depending on the setting and also the production timeline of whatever I am working on.

    For my books, I have shared entire chapters with people who are main characters, after they have agreed in writing to a specific set of rules for review. I email them the rules and ask them to email back consent, before I send the text. I can post those if anyone wants to see them.

    For stories (which in my past life were newspaper pieces, and now are largely magazine ones for mostly general interest magazines and some medical/science journals), I usually arrange a phone call by email and then go over by phone the part of the story in which someone is quoted or paraphrased. I also usually go over the lede/billboard of the story so they understand the context. At times, when the timing is tight, I have emailed chunks of text. Before the call or email exchange, I make sure the source and I have agreed to some ground rules.

    I also tried to do this as a newspaper reporter — and I would say that, as Trine said in her interview, I was noted for never having corrections — but newspaper timing for most reporters is much tighter than I assume it is for Trine as an investigative reporter. I invite anyone who doesn’t know the situation of newspaper science reporters today to imagine how they are supposed to find the time to do this when they have max 48 hours notice for a paper, and have to read the paper, look up the literature, find sources, conduct interviews, write a story, write an advance online post, and probably do a video piece as well. If you think they should have more time for that, please take it up with their industry.

    Now, how can this go wrong? The thing that people should keep in mind — and this is so obvious as to be unstated to the journalists among us, but may be an unfamiliar thought to the scientist science-writers — is that we are not stenographers. The purpose of our existence is not to render your research only as correctly as possible, but to tell a story about a finding that puts it in context. For that, we naturally assume a point of view (or “have an angle,” in industry jargon) and arrange the evidence in sequence so as to lead the reader through to the conclusion we have already reached. (NB, this is not the same as having an opinion, which is a larger and separate discussion.) It is entirely possible that the researcher about whom we are writing will disagree with this POV, as Deborah illustrates above. If they have the entire text of the piece, it is much more likely they will take issue with the POV as opposed to the accuracy of the description of their finding.

    This has happened to me several times; in one case in particular, a scientist felt his research (which is to say, himself) should have been featured more prominently, and launched a prolonged and vituperative campaign against me. In addition to being unpleasant, it was enormously time-wasting. So I don’t do that anymore, except, as I said, for books.

    Another issue that no one has really raised yet is the issue of fairness — which as journalists we should be bound by, even though we’ve all long abandoned the false fetish of objectivity. (I hope.) That is, if you’re going to share full text with someone, you really should be sharing it with everyone. If not, you have not treated all sides equally.

    And finally — if any of the editors have made it this far, they might comment — there is an issue of copyright and legality. If you are an employee or working under contract, the full text does not belong to you, it belongs to the publication. This raises a question not only of their in-house standards, but also of legal liability; under US law journalism is protected against “priori restraint” (attempted prevention of puboication) but prior restraint, and also defamation issues, become potentially more complex if the text has been circulated before publication.

    In short, which I was not, I think this is much more complex than many people assume, or than Twitter allowed.

  21. This has been a fascinating discussion to follow. At first it was really surprising to me that fact checking or meaning checking with sources wouldn’t be standard practice. I come from a social science research background though. Sharing your transcripts and interpretations of interviews with research participants is standard (even required sometimes) for ensuring that you have deeply understood what they meant. From a research validity point of view, what’s the point of an interview if you haven’t captured the participants real thoughts and feelings on an issue. Does this maybe fit more closely with Deborah Blum’s comment that science writers often do check?
    That said, the stakes are usually very different in research. The participants are almost always anonymous so reputations aren’t on the line in the same way (and if I’m being honest, hardly anyone reads the papers…). It’s interesting how much public perceptions of journalists and journalistic practices come into these decisions.
    Thanks David for sharing this and for this great comment section – very enlightening!

  22. Maryn says:

    Sorry, that should be “prior” restraint — tired fingers.

  23. Dirk Hanson says:

    Maryn writes: “The purpose of our existence is not to render your research only as correctly as possible, but to tell a story about a finding that puts it in context. For that, we naturally assume a point of view (or “have an angle,” in industry jargon) and arrange the evidence in sequence so as to lead the reader through to the conclusion we have already reached. ”
    —–
    Nicely put. This is a very hard concept for many scientists to get their heads around, and part of the reason why showing an entire MS ahead of publication is sometimes asking for trouble.

  24. But in this case, you will deal with this trouble/problem/headache anyway. That scientist will be upset once the story is published. So why not talk ahead of time, and ensure everything is airtight?

    I think with controversial topics, this is especially invaluable. I would rather hash things out before the ink is dry. I have found this sort of conversation is far happier/more pleasant when the story is still a draft than once the story is finished, and the source is angry and surprised and disappointed.

  25. I should add that I was joking in noting the low readership of research papers. That doesn’t make any difference in what should be considered good practice.
    And I wrote my comment before Maryn’s was posted and think it’s important for people like me to remember how different the structure of research writing and reporting are. As I once joked to Ed Yong (when he was kindly a research participant) “Don’t worry, I’m on a social scientist’s timeline not a journalists, so take your time”. I have a complete luxury to write and publish on my own timeline and at largely my own risk and ownership. That makes a huge difference.

  26. I should add that I almost never send anyone the entire draft. I’m not sure anyone has ever received the entire thing. Most often it is their own quotes, in context, with anything else I think they can help fact-check, which may be, in some rare cases, large chunks of the draft.

  27. Maryn says:

    I’m a little unclear on how these replies are threading (also, tired), so not sure if that’s a reply to me. Assuming it is, which might be wrong: I’m not much bothered by people being angry, as long as I’m factually correct; you could argue that making people angry is our job, actually. But in the situation I was referring to, the scientist who felt his ego had been insufficiently served did a lot of emailing to others along the lines of, “This person is about to publish an article that is unfair to me and I have tried to stop them and they won’t listen, please help” — which was deliberately inaccurate, but more to the point was much more complex to respond to than “This article portrayed my research inaccurately.” The larger point is that sources can’t always be trusted to be ego-free or behave like grownups, much though I’d like them to.

  28. Maryn says:

    I want to add one more thing and then I really, really do have to go do some paid work. Which is: Most specialty reporters I know do a kind of extra-publication fact-checking in which we recruit informal scientific advisors, sometimes to explain particular papers or processes, but often just to help us understand the field. Me for instance, I have people I can call/email if I need help understanding a knotty point in epi, immunology, biostat (shudder), trauma treatment, and so on. Depending on the situation, i might go over a piece of a story with one of them also.

  29. Mary Knudson says:

    This could be a good topic for discussion at next year’s Science Online and the annual NASW professional development day.

  30. “The larger point is that sources can’t always be trusted to be ego-free or behave like grownups, much though I’d like them to.”

    So true.

    But, for myself, I feel like going that extra mile with sources, even ones who may hate the end result, is worth it even if they start a campaign ahead of time (that has happened to me, too). That’s fine. Because I can always say, look, I even contacted you ahead of time, and I heard you out, and we discussed it and I am sorry that we didn’t agree.

  31. DeLene says:

    I agree Mary! It is fascinating to read about other people’s processes (thought and work), and their views on this issue.

  32. Maike Krause says:

    This has been an interesting discussion. I am a science writer in Germany, still in training, so I’ve only worked for two media outlets so far. But with both, a weekly newspaper in Switzerland, and a news magazine in Germany, it was common practice to send all citations with the surrounding text, even full articles to the interviewee for fact checking. That surprised me because I thought it might influence journalistic indipendence.
    Now I consider this an ambivalent matter. Of course, sometimes, with complex topics, I was glad that a scientist checked it for facts and corrected mistakes. But at other times it could be inconvenient or even change the tone of my article, for instance when the scientist canceled out some of the really good (well, from a journalist’s perspective, i.e. controversial or funny) things he said.
    My editors told me that letting the interview partner fact check (parts of the) text is a polite gesture, which is supposed to keep the sources happy I guess. But for a polite gesture there wasn’t much wiggle room for me if an interview partner objected to parts of the text that were correct but he didn’t feel comfortable with and therefor wanted them changed because in these cases most of the times the rule was: Do what keeps the interview partner happy.
    Now, working freelance from time to time, I always tell my interview partners that I will send them their quotes so they can fact check them (because editors will ask for that) but that I am not guaranteeing to adopt changes besides factual corrections. So far this has worked out ok.

  33. Yes, I totally agree that if you start worrying about making sources happy, you’re sunk. The goal should be to make sure the story is accurate and fair.

  34. So FWIW, here are my rules for mss review that I mentioned above, which I use primarily for subjects of book chapters, and occasionally adapt for use with stories. As I said above, if I am using these, I send them by email to the source/character, and ask them to reply indicating their acceptance before any text is sent.

    RULES FOR MANUSCRIPT REVIEW
    1. Please understand that I am sending this to you as a courtesy; the purpose of this review is fact-checking only, for maximum accuracy.
    2. To respect copyright, you may not share the text with anyone other than those who are named in it, or reproduce or republish it in any way.
    3. The ground rules for potential changes are:
    - If something is factually incorrect — e.g., wrong date, wrong name — and you can point me to a source that confirms the correct version, I commit to changing that fact in the text.
    - If a quote is correct according to a sound file or transcript, but you feel you misspoke, or feel you need to withdraw or change what you said, then I commit to having a conversation about it, but I do not guarantee that I will make a change. Allowing sources to change quotes is an extremely significant act for journalists, and so there must be a very good reason for such a change.
    - If I have described your emotional state or drawn a conclusion about events or circumstances, and you think my description is inaccurate, then I commit to having a conversation about it, and invite you to present any evidence that shows my construal is incorrect, but I do not guarantee that I will make a change.
    4. To respect the author’s contracted obligation to be the sole author of the text, please do not do any significant rewriting. If you feel changes are needed, your options are to:
    - use Track Changes to indicate areas you feel need to be changed
    - type the correct fact into the text in a different color or different font, e.g. ALL CAPS
    - write/type out the changes on a separate sheet, or put them on a new page within the Word file, using a “page break” to delimit them from the main text.
    5. Finally, it is important to understand that this text is a work in progress and may change further as it goes through edits. If the text changes significantly after our mutual review is done, I commit to letting you know about it, but please understand that my ability to change it back from the editor’s emendations will be limited.

  35. While I don’t ask sources to formally accept my terms before I send anything, Maryn’s rules are basically what I set forth to them, too. They are wise and work well, I think.

  36. Mark Henderson says:

    Really interesting discussion. I pretty much agree with everything Maryn has said here. Those ground rules are really interesting and an idea I might well adopt.

    A couple of further th0ughts, from a daily newspaper journalist’s perspective (I’m Science Editor of The Times in London).

    – Deadlines are important here. The need to have copy subbed and onto the page means that you often have very little time to incorporate any changes from fact-checking.

    – So too is the editing process once a story or feature is filed. It may be cut for length or otherwise edited, in ways I can’t always control. I thus can’t guarantee that the version I show a source will necessarily be the version that appears.

    – Pieces get held, for reasons you again don’t control — ie other news. This could be a problem if you’ve sent an entire piece to a source who doesn’t like it. This hasn’t happened to me, but there’s a concern they might get their retaliation in first — briefing their own version of events or spin to a rival paper, for example. Given UK libel law, there’s even in extreme cases a risk of injunction.

    – Overall tone can be an issue. I’ve sent whole pieces to scientists before who’ve pledged only to check quotes and facts, but have taken issue with the slant I’ve taken on a story.

    In general, I prefer to send sources smaller chunks of copy for fact-checking, for all these reasons. But I’ll make exceptions — particularly when the piece in question has a long lead time, and a complex narrative. Eg this piece I did on John Sulston and the genome project for the Wellcome Trust anniversary — http://www.wellcome.ac.uk/news/2011/features/wtvm051149.htm. Not much to lose and important that the whole thing was accurate.

  37. Dirk Hanson says:

    Yes, important to remember that daily deadlines can shatter all these good intentions in a heartbeat, and, as a journalist, you do what you can.

  38. “If writers aren’t prepared to stare down their sources over that principle, they shouldn’t be sending sources anything.”

    This is the key point, I think. On occasion over the years, I have selectively sent portions of articles to scientists for review — often to outside experts not sourced in the piece, but as John points out, sometimes the source is the only person who CAN verify some details. I always retained the right to decide which input to incorporate and which to leave out. In fact, doing so was standard practice at The Industrial Physicist magazine during its 10-year history. It was considered a courtesy. For other more general publications, I did so far less often, only when something REALLY needed clarifying, because it is generally not considered good journalistic practice.

    Honestly? It was a very rare occurrence when a source tried to abuse that practice and backpedaled on what s/he said — maybe 3 times over 20 years. It is far more likely to happen on longer, more investigative pieces dealing with contentious issues, though, so you need to be able to stick to your guns and stand up to a contentious source if you go that route.

  39. David Kroll says:

    There is SO much gold here from all of you. But I would be remiss not to reiterate Maryn’s passage above:

    I invite anyone who doesn’t know the situation of newspaper science reporters today to imagine how they are supposed to find the time to do this when they have max 48 hours notice for a paper, and have to read the paper, look up the literature, find sources, conduct interviews, write a story, write an advance online post, and probably do a video piece as well. If you think they should have more time for that, please take it up with their industry.

  40. Thanks, Maryn. These tend to be the guidelines I follow as well, but I’ve never taken the trouble to write it all out like that. Gacking for my own future use. :)

  41. Carl Zimmer says:

    Excellent discussion. Only thing missing is the matter of magazines checking submitted stories. It used to be quite common. Now that task often gets pushed on freelance writers to do their own checking. A change for the worse, both for the writer and reader.

  42. DeLene says:

    Are you willing to share how you deal with this added fact-checking in your own freelance work?

  43. Robin Smith says:

    Scientist turned science writer Robin Smith here. In my day job as a press officer, I frequently give talks to scientists on working with journalists. And every time someone in the audience asks: But why? Part of the reason scientists are so perplexed by this editorial policy is because the practice of peer review is such an inherent (if often bemoaned) step of the academic writing process — without peer review, a piece of writing isn’t considered
    trustworthy. Anything we can do to help these two groups of writers (academics and journalists) understand each other’s writing codes and ethics would be a help.

  44. I think scientists can teach journalists quite a bit (and vice versa). I think their reliance on data and evidence (and not gut feelings or false balance), is a great approach for journalism as well. I think their focus on what we don’t know is useful as well. And I think the peer review process, reshaped for our needs, is invaluable too.

  45. I remember reading that the New Yorker sent something like 900-plus questions to the Church of Scientology for fact-checking for its story. Amazing.

  46. For SELF (Conde Nast) and More (Meredith), I am required to submit a fact-check packet of *everything* I used for a story: transcripts, sound files, pdfs of journal articles, screenshots of websites, xeroxes of my notebooks. They go through everything, including checking quotes against the transcripts, and then they call everyone. It’s not only rigorous fact-checking, it often results in a better story; I have had editors say, “You didn’t use this, but I think that anecdote on page XX of this transcript makes a better walk-off than what you used…” SciAm does most of the same, though AFAIK they do not fact-check quotes with interviewees.

  47. So it sort of sounds like actually, this sort of fact-checking process is more common than we might have imagined?

  48. Pingback: Journalists, editors and science writers – checking with the source | Code for Life

  49. Grant says:

    I’ve finally found time to transcribe Elsie Hancock’s words; I’ve put them up on my blog as a post encouraging people this way with a few added remarks. Allow me to add them to the excellent conversation here:-

    She writes that she views science writing as a collaboration of sorts, with scientists presenting material she translates for a wider readership. Moving on to the subject of showing copy she writes (from page 46 of ideas into words),

    “And finally, consider the vexing issue of showing copy. This issue is always live, and more so for students.

    “Here again, the notion of collaboration helps you out. I usually say, “You will have the opportunity to fact-check, because I want it right as much as you do. And of course, I will be delighted to hear any other suggestions you may have about the piece.” The key word is fact-check. Beyond facts, there is no commitment to let scientists rewrite my words under my byline (as distinct from hearing suggestions), or even to literally show them copy. I do make an absolute commitment to get the material right.

    “On such a basis, showing copy or iffy parts of the copy can work very well. Do it in person, however. Sit right there, saying things like, “We’ll say X, then,” and leave with the amended copy. If you leave it, the scientist will get second thoughts, and you will be in big trouble. For short, straight-forward stuff, read the iffy bits over the phone.”

    (Emphasis as in original.)

  50. Adam Rogers says:

    If any of you would like me to send Wired’s (extensive) fact-check guidelines, which I send to all our writers, I’m happy to. It’s the document that explains what references and sources our fact-check department require to dive into a story, from the shortest to the longest.

  51. I would love to see it. Please do!

  52. I’m grateful to Mark for explaining, with much more care than I did, the particular challenges of fact-checking in a newspaper context. (Dirk also.)

  53. Well: Yes, but. Yes: Magazines (some, anyway, e.g. SELF, More, Wired, SciAm — there may be others) are definitely fact-checking. But: An important point to consider here is that this is being done by a third party, e.g., not the original writer. Which (in my experience) changes the entire transaction with the source, because the fact-checker and source don’t share the pre-existing interview intimacy that the writer and source do. As a result it’s more difficult for the source to exert the kind of pressure on the fact-checker that those of us who have had fact-checking go wrong have experienced as writers.

  54. Grant says:

    I’ll second that.

  55. This is an excellent discussion, and it has been an eye opener for me. The ethics that have been instilled in me over many years is that it is forbidden to show unpublished copy to a source and that getting approval for the speaker’s quotations is a violation of the professional standards of journalism. I go over every story I write sentence by sentence and highlight each phrase as I verify it independently. If that does not seem adequate I go, as Ed Yong suggests, to an independent expert. On a daily deadline that wouldn’t often be possible. But it doesn’t take much time to carefully paraphrase your own words and run that by the source, and follow up, if necessary, with email after email. Political and business reporters are also writing on deadline about complexities and they would never surrender an inch of editorial control.

    I won’t claim that I have not on occasion breached my own rules — and usually without bad consequences. (An exception was when Eric Kandel so helpfully offered to review a draft of a magazine profile I was writing of another neuroscientist.) I find myself agreeing with almost everything Maryn McKenna says. We are not conduits, and our job is not to avoid surprises. It is to inform and to surprise. I have also set different rules for my books, but that is tricky. Murray Gell-Mann has never forgiven me for not letting him read the manuscript of “Strange Beauty.” I feel confident that it is a better book because of that.

  56. Believe me, Adam’s guidelines are both exacting, and very, very useful. The system works because it’s redundant–the writer is responsible for getting everything right, and also for providing full documentation. And then a researcher/fact checker goes over it all again, including contacting sources and other experts. Having that third party–who is not sharing copy with the sources–involved helps keep the writer-source relationship untainted, and relatively pleasant, while also serving as a corrective for writerly prejudices and blind spots. That’s actually still the way it works at most of the magazines I write for, and a couple of them even still have a stable of experts they use to review everything that gets published.

    As George suggests below, it’s easy enough to replicate the process yourself as the writer, so long as you can free up a day to do it. And of course getting things right the first time around makes the fact checking a lot faster and easier. And yes, everything is tougher on a daily deadline, though I still would balk at sending anything more than a few sentences or perhaps a paragraph to a source, and then only with the explicit statement that you’re doing so to check for accuracy only. Those of you who do share copy regularly, don’t you find sources trying to finesse their quotes and such?

  57. If I remember right though, Hancock’s advice is for institutional science writers/PIOs, not journalists. Collaboration with your sources as a PIO is kind of built into the mix, and showing copy, I believe, absolutely standard. For a journalist, “collaboration” with a source would be an existential threat.

  58. Carl Zimmer says:

    Thomas is right. I just finished a piece for Wired and got massively hassled by one of their fact-checkers–which is what I love. I want someone to come to my story with fresh eyes, ready to find the blunders and contradictions that I cannot recognize myself. I started out at Discover as a fact-checker, and for a long time I was startled at how many errors could creep into the work of the country’s best science writers. Now I take it for granted. Along with Wired, I’ve worked with fact-checkers in productive ways at Discover, National Geographic, and Smithsonian. (I’m probably forgetting other places.)

  59. The thing is, at newspapers, we don’t have dedicated fact-checkers whose job it is to sift through our interviews, notes, papers and contact sources, etc. So there are two options – do it yourself or hope for the best. I opt to do it myself, with strict rules about how the process goes, very much like Maryn’s rules above. It has saved me many, many times from mistakes, both small and large, without compromising the story or my own integrity as a journalist. My stories often involve very contentious and controversial topics(www.chicagotribune.com/autism), and I doubt anyone would describe them as wishy-washy or written by committee.

    In investigative reporting, running parts of the story by sources before the story runs is common, if only to ensure the sources have a last chance to respond to any conclusions you have come to, and to ensure the story is airtight as it is likely to make waves.

    I think this is all good practice, honestly, if done with a firm stance on what is expected from both source and journalist. Or at least, it has worked very well for me so far.

  60. Carl Zimmer says:

    I’ve never understood why newspapers don’t have fact-checkers on staff the way magazines do, at least for the big stories like the ones you’re talking about. Must be some cultural difference that got frozen in place decades ago.

  61. It would be such a wonderful luxury! It seems incredible to me how few errors are made, considering the deadlines and the lack of the fact-checker safety net.

  62. Dirk Hanson says:

    Sadly, fact checkers/copy editors at book publishers are now more or less officially extinct. I remember really being put through the ringer as a first book author for Little, Brown. And lawyers would read it too for libelous passages. Now they won’t even employ editors to weed out the typos.

  63. Dirk,
    That is truly astounding to me, that book publishers have nixed the fact-checkers, and even the lawyers. You would think they would have huge incentive to get it right before thousands of books are printed.

  64. Thank you for writing about and discussing this! This discussion is crucial for the improvement of public science literacy, and for writers just starting out. When the scientific and public perspectives appear different, it allows the writer to find the communication connection pathway between both. Scientific discovery can seem meaningless if it is perceived as having no relevancy to the general public.

    Sometimes the findings anger the public, and sometimes putting the findings in a real world context can anger the scientists, apparently. The messages are still vital, and information is not effectively communicated if not voluntarily received by readers. Without being ‘falsely objective,” how can a writer strive to be as transparent as possible, thus building trust in the sources and readers? There is always a learning experience for all parties involved.

  65. Erik Vance says:

    Holy crap, what a thread! A lot of good stuff in here. Let me throw in a voice of caution. The whole idea of letting a source see anything makes me very nervous, especially with a long feature. Oftentimes a story holds some kind of tension, perhaps between two characters, in which case it’s our job to be the neutral arbiter. Other times, scientists just have a hard time seeing the forest for the trees and realizing that the reader is not picking apart every word like they would. To say nothing of their pride. The only two corrections I have ever printed have been related to titles and credit. We have to communicate accurately, but with specific characters and liberal use of perhaps inaccurate, yet sparkling metaphors. 80% of the time, it’s just not worth it and not ethical to run a short news piece past a source.
    As I read this, it strikes me that savvy readers should know that a daily news story is simply going to have small inaccuracies and sources should expect this as well (gross inaccuracies are another thing). If you want carefully checked content, read a monthly magazine, like Wired.

  66. I don’t agree readers should expect less from newspapers. Honestly, I think we (meaning my fellow journalists at the Tribune) strive to deliver accurate, carefully-checked content here, even on deadline. And I think every little mistake – even trivial – is a nick to our credibility, which is all we have. I don’t think readers give newspapers a break, even with small stuff – they see a name spelled wrong, or whatever, and they question the whole story.

  67. Dirk Hanson says:

    “they see a name spelled wrong, or whatever, and they question the whole story.”

    Brutal but true. I was recently involved with an Internet daily mag startup, and the lack of consistent copyediting–typos, misspellings, incorrect punctuation, grammatical errors–kept some early readers from taking the publication seriously: “If they can’t get THAT right…”

  68. True, true. It’s an inevitable reaction as a reader. I have that reaction as well! My names are often misspelled, and whenever that happens, I bristle and wonder whether it was a random mistake or whether the person is sloppy.

  69. Earlier a writer posted these words:

    “[Elsie Hancock] writes that she views science writing as a collaboration of sorts, with scientists presenting material she translates for a wider readership.”

    That’s not journalism then. Journalism is by nature adversarial. With science stories, it is usually friendly adversity, but the writer and source have subtly different agendas. What Elsie Hancock describes is public relations.

  70. John Rennie says:

    Yes. This topic is probably worth its own separate giant comment thread, but the sometimes unhealthy degree of cooperation between scientists and the journalists reporting on them deserves scrutiny.

  71. Dirk Hanson says:

    I knew this conversation would drift toward relationships with PR/PIO people sooner or later…

  72. Dirk Hanson says:

    As a journalist, I view my relationship to PR people and public/private information officers as essentially adversarial. Too often, their job is to keep me from doing my job. Or, at least, to guide me down a safe, pre-planned pathway. Establishing a working source relationship with a scientist or a CEO often involves cutting out the PR middlepersons. But both you and the source have to want it that way or it doesn’t work. I would never fact-check anything with anybody’s PR office, and I hope nobody else does, either.

  73. Meg Gordon says:

    Please don’t stop the discussion here…many in the hinterlands are getting their science stories from PIO written press releases dropped directly into their local papers with nary a word change. This for sure blurs the PIOI-journalist role for those who care and it affects the less erudite reader’s perception of science.

  74. Dan Vergano says:

    There are a lot of good and familiar ideas here, but one simple way to do this for a news story is to simply tell the source at the end of the interview what quotes you are likely to use. At least they know what is coming then, particularly on a contentious story. Sometimes they will distill the quotes into a better (or funnier) one, once they have heard what sort of message they are sending. This is about the only way I’ll check quotes with people on deadline. Always depends of course, reporting isn’t a science, thank goodness.

  75. Dirk Hanson says:

    “reporting isn’t a science, thank goodness.”

    Nice.

  76. Alan Burdick says:

    Thank you, Maryn, for posting your rules for manuscript review, which nicely codify the guidelines I’ve set out in similar situations.

    Like George Johnson, I was raised on the journalistic ethos that you never, ever show copy or quotes to a source prior to publication. When I write a story for a magazine or newspaper, that’s still the rule I follow. The reason is straightforward: the publication wants to be clear with its readers that (unless otherwise indicated) my article is not a “collaboration” between the writer and the source. I’m a hired gun, so my first obligation after accuracy is to respect that distinction. (I also want to be known as an independently minded writer, so I have a personal stake in that obligation too.)

    Still, I want to “get it right” myself, even before any on-staff fact-checker starts reviewing my notes and directly running facts past my sources. So before I hand the piece in, I typically pick up the phone and read (in paraphrased form) any relevant sections to my source(s) to verify the facts therein. This can get tedious for long stories. But I’m careful to have these conversations only over the phone, never through mail or email — to me that would violate the spirit of my editor’s law.

    With books it’s trickier. Trade-book publishers don’t employ fact-checkers, so the author is solely responsible for accuracy. And a book’s errors are harder to correct; you can amend in a second edition, but the nitpicky online reviews of the first edition will live forever to color other readers. Yet the scientific depth of a book — concepts and arguments that may unfold over pages — can be hard to fact-check strictly over the phone. I’ve taken an approach much like Maryn’s: I’ll show pages in advance, but with the clear stipulation that I want to hear about factual errors only, not matters of style or tone. Sometimes, as Ed suggests, I’ll show pages to a wise third party. But if I’m writing at length about the life’s work of a single researcher, ultimately there’s only one person I can run it past. Opening that door too wide can prove hazardous, and much depends on the source. Were I writing a book about Gell-Mann, I too would keep my pages to myself.

    As I’ve come to think of it, fact-checking is one part self-defense and two parts human kindness. By the time your story or book is published, you the writer are likely bored of it. But your very human source may just be waking to the realization that her thoughts and research, so ethereally discussed between the two of you, are now codified in print and in the permanent public record. One superb long-form magazine writer I know will read his entire finished story to the subject over the phone just before it appears in print. This catches last-minute errors, but mainly it serves to dampen the effect of “print-shock” on the subject, and it all but eliminates the angry letters borne of a participant’s post-publication remorse — all without compromising the story’s independence and integrity. It’s a little hard to pull off with a book, but next time around I may very well try it.

  77. I wrote a biology book targeting college bio 101 students for a publisher of a line of guidebooks. Anticipated the entire time that someone would fact check my work, in part because the author guidelines implied as much. Learned from the publisher at almost the last minute (last day or two) that, no, *no fact checking*(!). I sent out chapters that were particularly of concern to me to my own volunteer army of experts/fact checkers, who graciously reviewed them with rapid turnaround. But nope. Not from the publisher.

  78. Grant says:

    for institutional science writers/PIOs, not journalists

    Do you think so? Perhaps you’re reading that from her background; it’s not the impression I had of the text myself. My own impression was that her book was targeted at science writers in the widest, all-inclusive sense, which is not to say her suggestions might not be coloured by her professional background, as we all are.

    (I don’t think she means collaboration in quite the sense that your final sentence implies; perhaps that’s where your view is differing?)

  79. All very well put. For a magazine piece I want to be certain that when it gets to the fact checker there will not be a single mistake. Of course that never happens, and it’s humbling. For my last book and the one I am writing now I hired a fact checker. It seemed worth the expense.

  80. Grant says:

    It’s possible that people are reading too much into the word ‘collaboration’ here. I actually posted this material for how *similar* it was to points others here have made, not as being different (as people seem to be picking it out to be). My own reading of the word was simply a way of approaching people to put them at ease that they writer is keen to have the facts correct, not a collaboration over the text of the piece itself. (She makes quite clear the writer is in control of that.) I suspect it would help if I also gave the context prior to the passage I quoted, but my limited time isn’t up to transcribing much more.

    I’m not particularly a fan of PR work represented as science writing myself, that’s long story and another conversation…! ;-)

  81. The key, I think, is to be very clear with yourself about what the intent of the fact-checking is. It’s not to make a source happy. It’s not to appease anybody. It’s to make sure it’s all accurate and fair. That’s it. If you go in knowing that – and if you let your sources know that – this works just fine. I figure if you don’t have the guts to read stuff to them before publication, then you shouldn’t be publishing it in the first place.

  82. Actually, ‘guts’ isn’t the right word. I would say, if you aren’t comfortable reading it over the phone or sending it on the eve of publication, then you shouldn’t be publishing it in the first place…

  83. I sort of us this as a ‘gut check’ when I have written a draft. How solid what I wrote? Am I comfortable reading parts back to respective sources? If not, then I know I am on shaky ground. And this doesn’t mean I am trying to make each source happy. Not at all. It means is it rock solid? Is it so rock solid I can read it back to them and even if they aren’t happy, it is airtight? It’s a good way to tease out weak spots in the story, and to make those spots much stronger. Basically, on the eve of publication, I feel like I should be able to read anything back to anyone and have it be completely solid. If not, then I have more work to do. You know?

  84. Grant says:

    You’d laugh at this I know: when I do find time to cover work from the scientific literature on my blog, I fret over every little thing I write in a similar way to what you describe – so much so that the darn things take me hours. (I actually wrote a post asking others how long it took them to write a science blog post inspired by that!)

    Blogs are supposed to be a little more carefree (?!), and despite that this sort of writing doesn’t have the issue of sources in the quite same way as you do, I know what you mean.

  85. Alan Dove says:

    I’m very late to this discussion, but might be able to offer a slightly different perspective. These days I write almost entirely for trade publications and scientific journals’ news sections, so my audience is much smaller but also much more error-sensitive than what most reporters are used to. It’s a specialized niche. Readbacks are a standard part of my work, but with specific conditions.

    First, the source usually has to request it. Most of what I cover (biotech, microbiology, a little bit of public health, and related fields) is familiar territory for me, so I usually have a pretty good grasp on the concepts. If I don’t, I try to sort that out during the initial interview, or in subsequent discussions with the source before I’ve written the article.

    Second, I send the minimum amount of copy necessary for checking, usually just a quote plus the paragraph that contains it. That minimizes the opportunity for the source to try to edit my work.

    Third, I always specify in the email that I want them to check for errors. This is a discussion about facts, not tone or opinions.

    Fourth, after I get a reply, I always send back a note to the effect of “thanks for your feedback.” I don’t say I’ll make any changes they requested. It’s my name on the byline, not theirs.

    Finally, I always abide by my clients’ policies, if they have them. I have worked for a couple of publications that frowned on readbacks, and in those cases I told sources that they weren’t an option. I’ve also worked for clients (usually conference sponsors who’ve hired me to write meeting summaries) who insisted on sending the complete copy to all of the sources. That’s not exactly journalism, but it certainly is science writing, and it’s a case where fact-checking is so crucial that it trumps concerns about objectivity. I think of that more like a peer-reviewed publication.

    I’ll also second the earlier comments about transparency: whatever you do, be sure that your sources, your editor, and you are all on the same page, and assume that all of your discussions could become part of the public record.

  86. Hi Alan: Have you been saved from making mistakes? Even in areas in which you are very comfortable and have expertise?

  87. Alan Burdick says:

    Yes, very much so. Some mistakes were small and only visible to a specialist; others were small and slap-headedly obvious. Often changes involved helpful conversations about my interpretations of trends or more theoretical articles. But mostly I was surprised by how much I’d gotten right. I have no formal scientific training or expertise (beyond the college classes I took years ago), so I don’t have an area of comfort per se; everything makes me uneasy. I’ve convinced myself that’s a strength, as it pushes me to keep asking questions, assimilate, and build toward a macro view.

  88. Adam Rogers says:

    I volunteered to send Wired’s FC guidelines earlier, and a couple of folks said they’d like to see them…but I don’t have emails for you, and I’m not comfortable posting them someplace like Facebook (for no rational reason I can articulate right now). So email me first, if you want ‘em. first underscore last at wired dot com.

  89. David Kroll says:

    That’s a generous offer, Adam. Thank you very much. No need to post them publicly (for no rationale reason I can articulate either!).

  90. Pingback: links i like | b-scides

  91. I’m very late to this thread, but I’m really surprised at how many writers do send their copy to sources. I sent copy to a source once, as a green reporter, and got reamed out by my cantankerous British editor (a Guardian alum) who told me that I “should be ashamed” of myself and that I might as well be in PR. I’m not defending his harsh approach, but (perhaps because I’m getting to be older and cantankerous myself), I fully see his point now.

    As a writer or editor, I’ve never not been able to convince a scientist who wants to see copy that I will make sure to get the facts right in other ways. I think Ed Yong’s suggestion of having unbiased third parties vet the facts works 99% of the time. And for the other 1%, I’m with Brendan Maher.

    If you must check facts with the source, best to do it over the phone. That allows you to paraphrase, but even if you read a quote or even a para almost verbatim, they never get as attached to the words as they would seeing it in writing, and don’t try to tinker with them. Sending whole paras, let alone whole stories, to a scientist seems to me to be going down the very dangerous road of them wanting to make themselves sound smarter, cleverer, wittier. And I’ve seen this happen the handful of times that a writer who didn’t know my policy on this did send copy and then came back wanting to make changes requested by a source. In those cases, the writer did tell the source it was only for facts, but didn’t really matter.

    I also agree with whoever said that if you’re going to send to one source you should, to be fair, send it to all of them. I can see that for the run-of-the-mill story, perhaps there wouldn’t be much quibbling over quotes, but I can’t imagine having sources read over any of the pieces I’ve done on controversial topics before publication. I can still hear that cantankerous editor’s voice in my head now, telling me all the reasons that’s a total no-no.

  92. Mary Knudson says:

    The discussion continues. Look what you started, David. Good for you.

    I am coming at this somewhat differently than a few writers have expressed. It is extremely rare that I would let a source see a story. My purpose in letting anyone see a whole story before publishing would not be to get the source’s approval, not even mainly to check accuracy, but rather to get the blunt opinion of a knowledgeable physician who actually does the work I write about. I want an answer to this question: Is this story fair? A writer can quote 8 sources accurately in an article and yet the tone of the article may not be fair, may not have exposed the truth as best as a writer could.

    I am finally returning to such a lengthy story that I left hanging, when, for various reasons, I could not reach the type of source I was looking for, experts who were not cheerleaders for the medical strategy I’m writing about . To me there is no point in writing this story if I can’t write the truth as best I can find out. This is a story in the public interest and I want to write the pros and cons as carefully as possible to help patients and their families make an informed decision. I think on a long ,controversial and technically challenging story, a reporter can read so many studies and do so many interviews and get so close to the story that it can be in the public interest to have one or two knowledgeable experts with no vested interest in the topic read the story and express an opinion on whether, overall, it is fair and honest. As much as we medical journalists pour through studies, question them, do other interviews, other research, we still are reporters who lack the experience and intuition and street knowledge that doctors who routinely care for patients over many years have. So if my only purpose in publishing a story is to truly inform the public, why should I not take this last step and get the opinion of a couple of specialists who have worked with patients day in and day out for 10 or more years? The big salt debacle showed us that not everything doctors know and pass on to their patients comes from published studies.

    As for the other reason of checking accuracy, usually, I think, a reporter knows if she is unsure of some fact, not quite sure if the way she paraphrased something is true to what a source actually said. So that is the source she would go back to for an accuracy check. I see no reason to therefore feel I would need to check all a story’s quotes with all sources out of some sense of obligation to be fair to all.

  93. Mary Knudson says:

    Don’t know how the last three sentences of my prior comment got there. They should be deleted.

    [Fixed! - DJK]

  94. Another excellent post (and, yes, what a great discussion you started, David). I try to hew pretty closely to what Mary describes. On a difficult story you find experts who are not sources to give it a look. If you’re writing about familiar territory, you know which material can only be verified with a particular person and which is general knowledge. And even then, most fact checking, nuances and all, can be done through paraphrase. Sources should see the story itself when the rest of the world does.

    Several people here have said that it is especially important to run quotes by sources. To me that is the last thing you want to give them verbatim. We all recoil at the sound of our own voices. If an interview was on the record (the default assumption) then trust your notes or your audio recording. Even experts can slip and misstate something they know full well, so paraphrase their words back to them to be sure the semantic content is correct. An exception might be when something was said or written in an email when the ground rules were uncertain — especially if the source is not a public figure or someone used to dealing with journalists.

    After this discussion was under way, I asked a former boss at the Times, who is now a deputy managing editor, for his take. He said there is no ironclad rule against showing part of a story to a source to verify accuracy and precision as long as the ground rules are clear — editorial control remains with the reporter and editors. And, he noted, it depends on the relationship with the source. You get to know who you can trust.

  95. This is a great discussion. I also fact check, especially on long pieces that I may have been researching for months, then synthesizing the comments of many into a narrative theme. I don’t share unpublished manuscripts with sources or people in the field but I do fact check with individual sources, particularly on nuanced passages. Like Mary Knudson, I feel no obligation to fact check with every person quoted in the story. For the most part, my method is to paraphrase. If it’s an arcane scientific description, I may read a phrase verbatim to make sure I haven’t messed up an important definition. This formalized fact checking has helped me catch errors and also, at times, has led to follow-up discussion with the source that sharpens my thinking and may lead me to recast some summary sections. The story is better for it.
    On long pieces researched over many months, I also think it’s fair to let people know what parts of the interview I’m including in the piece. I’ve got my notebooks; they sometimes hardly remember the conversation.
    With official fact checkers, I’ve had mostly good but occasionally not-so-good experiences. One recently really messed up my research field by inserting their opinions in conversations with my sources. So while I’m a fan of fact-checking in the name of accuracy, thoroughness and fairness, I have to say there are some bad players out there.

  96. Kausik Datta says:

    I am very late to this discussion (I am properly ashamed to say), and I also feel kind of awe-struck in presence of the luminaries of science-writing that I follow through various media – such as Trine, Maryn, David Kroll, and others. Therefore, please pardon my naive question – did Trine not mention ‘Fact-Checking’ in her podcast? I mean, she – as a science journalist – is interested in making sure of all the facts that she represents in her column/article/write-up. Is that not a laudable, and essentially desirable, objective?

    I seek to make a distinction between ‘fact-checking’ and, for want of a better expression,’interpretation-checking’. Facts are incontrovertible, are they not? So when Trine goes back to the source, all she is asking is the confirmation of the facts as already presented by the source. She is not asking for additional opinions, ideas, interpretations, arguments, discussions or any other form of discourse – as far as I could tell. So where exactly is the scope for introduction of a bias, on part of Trine (or any other science-writer) in this process?

    I apologize once again for my ignorance and lack of understanding of this issue.

  97. David Kroll says:

    Kausik, no, you are correct. Trine specifically used the term, “fact-check” at about 13:38 of the interview. She makes the point that the value has been that sources have caught subtle differences in her representation of the science that may not seem important to the non-expert but that would make a big difference to scientists (my paraphrasing). I do agree with you that this is more, “interpretation-checking,” but my pro journalist colleagues here would likely tell you that this falls under the general heading of fact-checking.

    btw, I’m honored that you would put me in that list of “luminaries” but it is I who is the student here – hence my motivation for putting this post together in the first place.

  98. What a discussion!

    Just to be clear, I do this for fact-checking purposes only. It’s not to get approval from my sources, or anything like that.

    Here’s an example of how sources can help make a story more precise by pointing out something subtle: I first wrote about the retrovirus XMRV and CFS back in June 2010 (http://articles.chicagotribune.com/2010-06-07/health/ct-met-chronic-fatigue–20100607_1_chronic-fatigue-syndrome-xmrv-autism). I’m not a retrovirologist, and before this I had written almost nothing about PCR or retrovirology. I knew almost nothing, to be blunt. In my earlier drafts, I kept writing that Lombardi et al had found XMRV in the blood of CFS patients…etc.

    Well, that’s not right. They found evidence of XMRV in the patients’ blood, not XMRV. And my sources had the opportunity to alert me to that because I sent them chunks of my story – not the whole thing – for fact-checking.

    If I had stuck with my original draft, it would have sailed into the paper and likely nobody would have called me to say, hey, that’s not really right.

    But, I still would have been wrong. And to scientists, I think, I would have lost a little bit of credibility by being imprecise in that way.

    This has happened to me many times, not because I am sloppy or stupid but because I’m not an expert in any of this and no amount of interviewing will turn me into one. So why not enlist sources in fact-checking? It’s like trying to learn a foreign language in a few days, and then turning to native speakers to check your grammar. You’re not asking them to speak for you, you are asking them to ensure you don’t sound foolish.

  99. Greg Weatherford says:

    I’d love to see them too. Would you mind? I’d like to use them in a magazine-writing course I am planning at Virginia Commonwealth University. I will send an email …

  100. Pingback: TWiV 150: Contaminated

  101. Pingback: How do scientists view fact-checking by science writers? | Take As Directed

  102. Kausik Datta says:

    If I had stuck with my original draft, it would have sailed into the paper and likely nobody would have called me to say, hey, that’s not really right. But, I still would have been wrong. And to scientists, I think, I would have lost a little bit of credibility by being imprecise in that way.

    This is why there needs to be more of fact-checking, not less. I greatly admire this attention to detail and precision, Trine.

    Thanks to David for making the distinction between fact-checking and interpretation-checking a lot clearer than I could. If I were a science journalist (I wish!), I would certainly do the fact-check as Trine does, but would stay away from interpretation-checking. My interpretation of the data (facts, if you will) would be mine, reflecting the way in which I understood the work – with the proviso that interpretations may, and will, sometimes differ. Therefore, having a source (or even any other third party) check my interpretations may introduce an element of bias.

    I say this from the point of view of a working researcher as well. When consulting published, peer-reviewed papers, we often come across papers where there may be alternate (even broader or narrower) interpretations of the data (this is greatly facilitated by the availability of supplemental data in, say, the Open Access journals). We are given to understand that this is acceptable, that a complete picture often emerges only after reasons synthesis of multiple interpretations.

    So, my question to Trine and the others is: are the science-writers and science-journalists to be held to a higher standard? Are they expected to offer definitive pronouncements based on the data, rather than represent the data as they are – warts and all?

  103. Yes, this continues to be an interesting discussion. I don’t think anyone here is arguing against checking facts with sources. That’s fundamental. The disagreement is over whether it is appropriate to do so by showing the source an entire story before publication. In my experience that is highly unusual and has always been controversial.

  104. Stephen P. Shaw says:

    I did not mean to ‘injury’ some. My observations are my own thoughts and feelings. I do apologize for overstepping journalistic bounds, but seeing as I don’t have any history where that lies; I am ignorant of the protocols. I realize that now having more sensitive info, requires the writer to disclosure issues. Again, I apologize.
    I do believe however that religious issues and beliefs in regards to free speech are the sole property of the writer. Any thoughts to this regard, I do not feel sorry for.
    I hope to be in touch, and I will be reading & learning with enthusiasm.
    Thank You

  105. Pingback: This Week’s Blobologist Approved Reads | The Blobologist

  106. Pingback: Should science journalists check copy with their sources? « Quomodocumque

  107. Pingback: Scientists should not be allowed to copy-check stories about their work – Daily Manzar Namaa

  108. Pingback: Scientists should not be allowed to copy-check stories about their work Extensive copy-checking by scientists before publication does not serve journalism, science or readers « untoldstoriesinternews

  109. Chris Barncard says:

    It’s been my experience, like Jennifer’s, that sources rarely try to change the meaning of their original comments.

    But in touchy situations it was also my practice — especially if a source asked me to read back quotes or explain what I heard them say — to let them know I’m confident in my note-taking abilities. That is, I know what I heard, and our interview progressed based on your answers to my questions. If you decide after the fact that there was a major misunderstanding, it will take a strong case to make me doubt my notes.

    When I was a newspaper reporter, I did not want to share material with sources. It just felt wrong. But that was the only reason I could come up with to deny requests from interviewees who wanted some insight into what I heard them say. And it’s a pretty weak one. And very hard to defend.

  110. Pingback: Scientist copy-checking: Point-counterpoint at the Guardian | Take As Directed

  111. Pingback: | Terra Sigillata

  112. Pingback: Guardian ends run of smart science journalism discussions, publishes exercise in idiocy | The Panic Virus

  113. Pingback: “To inform and surprise” | genomeboy

  114. Pingback: Scientists vs. Journalists: A Field Guide | Deep Sea News

  115. Pingback: Freedom of the Press: Science Reporting Division - NEWS AND ARTICLES – NEWS AND ARTICLES

  116. Pingback: Introducing SciWriteLabs. Today’s installment: Kroll and Racaniello discuss the journalism/factchecking debate | The Panic Virus

  117. Pingback: SciWriteLabs #2: Reuters’ Ivan Oransky and Wired’s Adam Rogers on sources v. subjects & more on the factchecking debate | The Panic Virus

  118. Pingback: Jekyll » Blog Archive » A chi spetta l’ultima parola prima della pubblicazione: giornalisti e scienziati a confronto

  119. Pingback: SciWriteLabs #5.1: Nature’s Ananyo Bhattacharya on “Bad Science,” inverted pyramids, and information webs | The Panic Virus

  120. Pingback: Cientistas devem revisar notícias de ciência? |

  121. Pingback: It’s not journalism, but maybe it’s not completely evil, either | bench twentyone

  122. Pingback: Science Writing Sampler Platter | Take As Directed

  123. Pingback: Cientistas devem revisar notícias de ciência? | Do Paper à Pauta

  124. Pingback: Thoughts on academic scientists giving media interviews | Take As Directed

  125. Pingback: Science reporting: Covering the environment, technology and medicine – Journalist's Resource: Research for Reporting, from Harvard Shorenstein Center

  126. Pingback: Spillover and science communication