#Overly Honest Methods or PhD Madness?

Bookmark and Share

This week, scientists took to the social networking site Twitter to swap candid examples of what really goes on behind the lab’s closed doors.  Their confessions, under the Twitter hashtag #OverlyHonestMethods, are becoming a rapidly growing collection of humorous anecdotes that reflect the realities of scientific decision-making, particularly within the life and physical sciences. Overall, it reads like a catalogue of methodological misdemeanours.

Although it has since spread to Facebook, blogs and other sites, as with all social media trends or “memes”, the popularity of #OverlyHonestMethods is likely to soon die down.  However, I cannot help but wonder whether, after the dust (and amusement) settles, the status of, and public trust in, the scientific method will have been challenged.  On the other hand, this surprising candour might actually be good for science.  Either way, the scientific community needs also to have a more serious discussion about this topic.

First, let’s have a look at some examples.  One post – or “tweet” (in Twitter terminology) – sees a scientist confessing that she “selected a particular qPCR [real-time polymerase reaction] mastermix because the rep was giving away legomen USB sticks”!  Admittedly, I myself am a sucker for freebies.  Goodness knows I have signed up to far too many things that I might otherwise have avoided on the basis I was suckered in because of a rep offering a free mug, or indeed a novelty USB stick.  As a social scientist, I don’t have to buy reagents or chemicals.  However, if I did, I would at least like to think that my choice was based on a well-informed assessment of the suitability of all the relevant reagents available to me.  Not (necessarily) the one that comes with the coolest USB stick.

Whilst posts like this one hint at the some of the more arbitrary choices that lab scientists sometimes make, other are posts are more alarming to me, because they hint at some of the more unprofessional and even “unscientific” things that go on in the lab.  For example, one tweet alludes to the fact that “data are available on request because then we can tidy the spreadsheet only if absolutely necessary”.  Another claimed that his “experiment was repeated until we had three statistically significant similar results and could discard the outliers”.

I guess the worrying thing about #OverlyHonestMethods is that they may be just that – overly honest!  There aren’t that many posts which seem so incredulous so as to be fabricated entirely.  Of course, it’s their very plausibility that makes them funny … to other scientists that is.

However, if even a proportion of these “trade secrets” are indeed true, then we need to stop for a second and ask ourselves what implications might they have for how the standards of the scientific method, which are currently so crucial to the epistemic authority of science?  Science is, and always has been, built on its foundational principles of objectivity, reliability, replicability and validity.

I wonder how many journal editors would see fit to consider a retraction if the information submitted to Twitter was submitted directly to them, and it related to a paper they had previously published?

This consideration leads me to a related question: how will these confessions be received by the public, who generally speaking, trust that scientific research returns objective results precisely because it adheres to its principles of reliability, replicability and validity?  Of course, whilst most non-scientists are unlikely to get many of the in-jokes in these anecdotes, neither are they likely to find funny the fact that scientists are alluding to questionable use of their research funds.  Particularly since many of these scientists are likely funded by the public and/or state funds.  As one tweet claimed:  “Functional magnetic resource imaging was performed because we had to justify this large grant somehow”.  Although one can assume (or hope) the tweeter was joking, in a post-recession climate where resources are strained, many may fail to see the funny side.
Maybe #OveryHonestMethods is simply a group of scientists arguing that the scientific method, like anything in life, isn’t as perfect as it is sometimes made out to be.  After all, scientists are human, and humans make decisions that are sometimes less-than-objective.  This wouldn’t necessarily be a new argument. Sociologists of Science, like Harry Collins and Bruno Latour have long been making the argument that natural science, including its methodological decisions, is shaped by social, political and cultural factors and interests, pretty much like any other activity or practice.  The historian of science Steve Shapin discusses to what he calls “idealized methods stories”.  This is where the contingencies, uncertainties, ambiguities and all-round messiness of the reality of decision-making in the lab are hidden from the sanitized end product – the published article.    The big difference with #OverlyHonestResearchMethods is that it is now the natural and life scientists themselves who are making this point.  And for all the world to see.

Costs of Candour?

Illustration: Mad Scientist source: Wikipedia

Illustration: Mad Scientist source: Wikipedia

 

Who knows, this candour might ultimately be good for science in the long run.  It might finally serve to make visible the hitherto “behind-closed-doors” practices which have always been a part of science.  Maybe many of these examples are necessarily and realistically a part of what makes science science?

 

 

 

 

Maybe we have traditionally held science to unrealistically high standards that it cannot meet?  Maybe we need a reality check.

It might however also represent the very beginnings of a “revolution” in how scientists and the public alike, view and talk about the scientific method.  Stranger things have happened as a result of social media!
Here in United States, some consider the website Wikileaks to be a threat to national security.  One could go so far as to argue that #OverlyHonestMethods is a threat to scientific security. In an age of evidence-based policy and practice, the legitimacy of many policies are themselves tied to the legitimacy of the science upon which they are based.

For the time being, one thing I can be sure of is that, if I were grad student, or a scientist seeking a job, I would think twice about posting any of my methodological misdemeanours.  In a time when competition for jobs in science is tighter than ever, publicly admitting to “pushing buttons in our favourite stats software until all our results had stars next to them” or “this dye was used because the bottle was within reach”, might not be the wisest decision in the world.

Dr. Simon Williams is a Research Associate at the Feinberg School of Medicine, Northwestern University, in Chicago, Illinois.  As a member of the Scientific Careers Research and Development Group, he is interested in issues of scientific training and careers, as well as broader issues related to scientific knowledge and the scientific method.

 

Related Posts Plugin for WordPress, Blogger...

Creative Commons License
#Overly Honest Methods or PhD Madness? by PLOS Blogs Network, unless otherwise expressly stated, is licensed under a Creative Commons Attribution 4.0 International License.

This entry was posted in PLoS. Bookmark the permalink.

38 Responses to #Overly Honest Methods or PhD Madness?

  1. David C Logan says:

    There will always be those, like conspiracy theorist, who simply can’t take events at face value. It was and is a bloody joke! Don’t overly analyse and pervert it just because you can. Let’s not forget these are tweets, not confessions, and not scientific papers that are reviewed and have to stand up to the tests of time and reproducibility.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  2. smidget says:

    I also imagine that the vast majority of these are jokes. We’ve all thought “ugh, that dye is on the other side of the lab… this one is right here… hmm” but then didn’t do it. Or “oh hey, this statistical test is significant! But damn, it totally misrepresents the model. Oh well.”

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  3. Amy says:

    I found this article through people calling you a party-pooper on the #overlyhonestmethods tweet stream. However, it puts into words exactly what I’ve been thinking over the last few days – and why I love the meme so much. Also a social scientist, and one who works at a marine lab surrounded by biologists and chemists, I have echoed Collins and Latour’s points in both my academic work and regular interactions at the lab. Yet, until now it’s fallen largely on deaf ears, so it’s nice to see the realization that scientists are human and science is culture emerge from practicing scientists themselves. I think it’s a fantastic step forward for society.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  4. Bob says:

    You seem to base your argument on the premise that these confessions hint to some major flaws in how science is being done these days, how lack of “reliability, replicability and validity” are endangering the scientific method. You are completely wrong in thinking that if all these “#OverlyHonestMethods” are being written down in the researchers notebook… that is exactly what leads to”reliability, replicability and validity”. It is serendipitous mistakes like “using a free reagent”, “incubating for 24 instead of 4 hours” or “used this dye because it was the closest at hand” that lead to some of the great breakthroughs in research. And as someone supposedly concerned with “scientific training and careers”, you should be able to understand the time constraints that would make someone “tidy the spreadsheet only if absolutely necessary” which does not mean falsifying data, it means thing like putting labels on cells that you know what the function is computing but someone else looking at your spreadsheet would not.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  5. Graham Freeman says:

    Either a very dumb assessment of the meme, or a very clever troll.

    </notsurefry>

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  6. Simon Williams PhD says:

    Many thanks for the above comments. As hoped, the blog post has elicited a range of views and perspectives. Hopefully my post was clear that there are a number of questions to be asked in regard to #OverlyHonestMethods, including the consideration of whether it is a good or bad thing. Either way, it means something. It raises interesting questions about how science is really done. As such, its importance extends beyond the twitter hashtag within which it was conceived.

    To the sociologist of science (which to reiterate, is my disciplinary background), it represents an interesting development. As the post notes and as Amy comments above, the claim that science is driven by what we can see as social, cultural and economic decisions and factors has, in the literature, been rejected by positivist accounts of science (which were espoused largely by scientists , since at least the late 1980s). What we see here are great examples of the micro-politics of the lab. For anyone who hasn’t read it, I would recommend Harry Collins’ and Trevor Pinch’s book ‘The Golem: What Everyone Should Know About Science’. As the title suggests, it is a good intro to the sociological study of scientific practice and knowledge.

    My interpretation of the #OverlyHonestMethods is that it relies on a “its-funny-because-its-true” type of humor. I agree it is important to note that a portion of these are likely purely made up (a point I make in the blog). However, a number of these “in-jokes” have to have at least some truth in them, otherwise they would fail to be amusing in the way that they were intended. In-jokes are a form of “inside knowledge”, seldom understandable to outsiders. As I mention, such in-jokes would be generally lost on the non-scientist, but I think there is still one general interpretation which non-scientists might still garner from this: that science isn’t as objective as it has traditionally and officially purported to be.

    I did want to point out also that I am not arguing that my post is “hinting at some major flaws in how science is done these days how lack of ‘reliability, replicability and validity’ are endangering the scientific method”. Although I have read about it via the historical works of scholars like Collins and Latour, I myself haven’t been around long enough to directly study how science used to be done. I will leave it to scientists and sociologists who have been around long enough to comment on how science used to be done, to pass judgment on whether scientific standards are different these days. My aim is to raise the related questions of “has science always been subject to the kinds of behind-the-scenes practices discussed in #OverlyHonestMethods?” and if so “is it a good or bad thing that we are being so honest in this way?” My leaning is that, in the short term it carries some negative implications, but that in the long term, discussions of this nature could actually be good for science.
    I acknowledge that these are tweets and not journal methods sections.

    However, the question of research ethics is also brought into play. For those contributions that do have some truth in them and which do allude to scientific unprofessionalism (as it has been traditionally defined), isn’t there technically some ethical implication involved?

    Also, philosophically-speaking, some of these scientific standards (as they are traditionally defined) would indeed be violated or jeopardized by some of these examples. Validity is the obvious one. But let’s take replicability for instance. If we did literally write in our article’s methods section “I chose the reagent which was closest to me” or “I picked the chemical which came with a free USB stick”, then the next scientist trying to perform the experiment would likely choose a completely different reagent (whichever was nearest to them in their lab or whichever came with a free USB this time). This is of course a philosophical and hypothetical point. However, the point about being a little cautious in having your name and Twitter profile attached to a tweet which IMPLIES (irrespective of whether it is actually true or not) you or a colleague has fallen foul of IRB or ‘official’ methodological standards, is for me, a practical one.

    I’m not saying that I don’t, to some extent, acknowledge and understand the exigencies and social realities of scientific research and practice. What I am saying is that #OverlyHonestMethods is the first time that, to my knowledge, physical and life scientists are explicitly coming out and revealing them. And this carries with it the need for critical discussion amongst scientists and those with vested interests in the products of scientific research.

    Thanks again for your comments. I hope the PLoS Blog Community keeps them coming.

    VN:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • David C Logan says:

      Now you are being a little silly.

      VA:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
    • Oliver says:

      Wow, you’ve just given a textbook example of overthinking something. You clearly don’t work in a lab as you seem to have completely misunderstood the nature of these memes.

      VA:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
  7. Pedram Razghandi says:

    I think this meme is something we rather need. It’s entirely a positive good to hold ourselves to a very high standard of rigor, since we are after all trying to explore the natural world as accurately as possible (and if the results will have applications for human technology and medical use, rigor is if anything even more important). It’s a funny meme precisely because all of us (and I mean all of us) have at some point witnessed or even engaged in at least one of those shifty-eyed , ahem, ‘labor saving methods’.

    Will the public become disillusioned by our honesty, when they find out we have feet of clay? Perhaps, but it is something so many of us brought on ourselves. On the other hand, it shows that while the natural world is an objective thing, the pursuit of science is a human endeavor–we’re not magic ‘super-geniuses’, just regular people who happened to fall in love with some aspect of the awesome thing ever, i.e. nature, and had the right educational opportunities/financial support to pursue its study as a career. Our jobs often involve #grindingit, using what we have at hand to do the best we can with our experiments (a lot of my nonscientist friends are surprised at how blue-collar our labs and procedures are; they usually expect something slick and futuristic, and exactly the right reagents for the occasion carefully stocked and ready to go). If anything, that sounds like a need for more funding–certainly more efficient funding–for the materials themselves, and less corruption.

    I think the frauds do more to disillusion the public. But seeing us act to fix these problems immediately (and maybe reforming the corrupting system of the academic publishing industry…) and clean up our act will probably win people back with more enthusiasm than ever.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  8. JLS says:

    I agree that some of these are “funny because it is true.” But in what way? Funny because it is true that I did it, or funny because it is true that I thought about it before or joked about it before. A lot of amusement is because of the latter.

    Also, sometimes researchers try funny things for funny reasons initially, but the rigor and good reason does later follow when you decide on a path of action.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  9. Leigh says:

    The thing here is, we do report the science as it happens. This is not really being called into question. The meme lampoons the idea that there is a good, a priori reason for every action one takes in the lab. Sometimes we don’t necessarily have a scientifically well justified reason for proceeding as we did, and sometimes we tried shit until we could prove the assay worked. I posted my original thoughts on this and i don’t see it having ventured too far from that general line of thought.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  10. SM says:

    Simon, quick point, having read a majority of the overly honest methods, I can say that very, very few of them actually flirt with misdemeanor. Take your example of qPCR for example. I understand that this may be a subtlety lost on a social scientist, but a lot of methods in the basic sciences have a set “recipe”. It doesn’t really matter which company you buy your reagent from a lot of the time, because a lot of them are chemically identical.

    I agree with what a commenter said before. I think a lot of the methods sound like an exaggerated version of things we wish we could do. Or mistakes we may have made while learning a technique but subsequently corrected. And having a laugh about it.

    Given the lack of any genuine science humour, outside of portraying scientists as social rejects, I think this level of hand-wringing is very unnecessary. I think you’re wrong in saying that a lot of these are in-jokes, the premise of the humour is very approachable even if one doesn’t know the steps involved in PCR.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  11. David Martin says:

    “We don’t understand, we only do – welcome to the kit generation” was written on a sign in a lab in Oslo in the 1990’s as commercial kits made the job of doing gene manipulation easier. The memes are self mocking – they reference the thought processes that go into selecting and performing appropriate experiments. It’s not so much a confessional, as a reductio ad absurdia of the cargo cult approach to science. We see it in all fields where over-reliance on a trendy method without really understanding it, or fleshing out a weak argument with the veneer of erudition has it’s pomposity or naivety punctured by acerbic humour.

    It is a reflection that we do not always understand every detail of the methods we do, that there are social and other pressures, tradition and folklore in the way that the experiments were done – I once timed assays with Barenboems recording of Ravels Bolero – a happy coincidence and a lot nicer way to time it that an electronic pinger.

    I’d be more than happy to consider those who can make such jokes, becaue you cannot make them without understanding. They are ‘in’ jokes which is why your piece falls so far outwith the domain to which it purports to purtain.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  12. Theresa says:

    Hello Simon,

    Thank you for putting together this blog post. I personally share some similar concerns (I wrote a little bit about this on my blog).

    Researchers face decisions everyday – some of them are arbitrary and are matters of convenience – that’s fine to me as long as no scientific integrity is affected. Things like what cell line to use, what reagents to buy, leaving things in water bath longer than needed (as long as the reagent isn’t degraded), they happen. So for these I am not as worried about them – from a scientific point of view.

    However, I have seen many tweets that really bothered me. Things like “I drop three data points because the other half of them were made up anyway” just gave me chills. This is not funny – this is simply bad science. And there are quite a few like this.

    In addition, regardless of what are posted, what the intentions are, we should still keep in mind that all those posted are available for the public to see. Sure, they are mostly “inside jokes.” But there is nothing that comes with the hashtag to say “these are mostly jokes.” Many actually talk about it as a display of how “real science is done” – now that’s a problem – especially when these tweets are out of context. As funny as #overlyhonestmethods tweets are, in the current time when research fundings are cut and many don’t trust evidence-based science, I don’t think I can enjoy these tweets without some mixed feelings. (I have worked in science outreach for the past 8 years and spent 5 years in research)

    But I am starting to look at this from another angle. This gives us a rare opportunity to look into issues in science and ask “why” – why do people need to use free reagents (lack of funding), quote abstracts and not read the papers they refer to (paywall – open access), drop data points thinking that it is okay (training in scientific methods). So in the long run this might be good for us.

    Anyway, thanks for writing this. I feel less alone about holding a similar point of view!

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  13. Pingback: #overlyhonestmethods – Funny? Not so funny? (updated with postscripts) « Science, I Choose You!

  14. Tl;dr.
    Lighten the heck up.

    #overlyhonestcounterpoint

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  15. Roget says:

    It’s funny because it could have been true but nobody did it but everyone ubderstands it. We always joke in the lab about we could do this and that so it is funny but when it comes to work it is strictly professional and we follow the strict scientific method to show our findings. Nothing more nothing less. I am sure thawing the reagent in the bra actually happened lol. Cheers

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  16. Gareth Walker says:

    If scientists speaking honestly about how they conduct themselves leads to less public trust in science, then that shows that currently the public trusts the systems of science more than it should. More honest and accurate information is always a good thing. Scientists should never try to inspire faith.

    There has been a worrying trend recently for non-scientists to develop a love of science, and see the claims of scientists as more valuable than they are. Much better to have scientists toiling away as the unloved egors of society.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  17. sciliz says:

    You know, I know this is a crazy idea, but you could actually ask us what our intents are. Just because we, as scientists, are the mere subjects of your discipline, and not the erudite (read: stuffy) “objective scholars studying the practice of science”, doesn’t mean we don’t think about these issues too.
    Granted, I don’t speak for all the people who contributed, but I think there are least three fairly obvious angles on the sociology side of this meme that you’ve completely missed.

    First, a good chunk of the tweets (including the one of mine that got, by far, the most attention- about shaking tubes “like a polaroid picture”) are pure silliness. That is not to say that there is not a real social purpose served by such humor- humanizing the face of science is itself a valuable pursuit. It’s why a great many of the people I tweet with are on twitter, and it’s something I consider as well.

    Second, the next most popular tweet of mine was one for which I had a specific socio/political agenda (it was tagged #watson and #crick and it concerned their “method” of yoinking a picture from Franklin). This was specifically aimed at mocking sexism in science. Or at least sexism in the stories we tell about science (including the story of the authorship on papers).

    Third, many of my tweets (and several of the ones I most enjoyed from others) were extremely detailed and the practice of science could actually be greatly improved if we did include info like that in methods sections. Are they in-jokes? Sure. That doesn’t mean that the point is a “mere” giggle, though. For example, one of my tweets concerned the specific polymerase you have to use for a PCR application. As I learned from a postdoc in a lab down the hall (only after struggling for some time), you might have a heck of a time trying to amplifying Plasmodium DNA if you use the proofreading polymerase Pfu. Use Taq, and everything turns out fine. Why? Well, we could speculate about this perhaps having something about AT content bias (the Plasmodium genome is 80% AT), or maybe it’s something about proofreading, but really? We’ve no clue. However, some poor hapless grad student should not have to discover that stuff every. frickin. time. As far as I can tell, that kind of thing wastes FAR more resources than an fMRI study aimed at “glamming up” some otherwise sound science to get it more attention and keep a funding stream going for a solid lab. Do these bizarre technical nitty gritty findings get discussed over beers at conferences all the time? Yep. Does that state of affairs systematically disenfranchise scientists who are not the well-connected conference goers? Yep. Can the internet democratize knowledge of all kinds, including how to actually conduct experiments? That would be the hope. Granted, I have no idea what effect that would have on sociology, but I think it would be good for science.

    As an aside, as a sociologist of science, I’m sure it has not escaped your attention that you yourself used a cartoon (dehumanized) male (sexist) image to represent the “scientist” you were discussing in your blog post. Sociology of science is, in it’s own right, a fascinating discipline. But some of us would rather be the change we want to see. Be cautious about ascribing motives (or lack thereof) to other people, merely because they are less formal than you.
    Overall, I feel kind of sorry for you. It must be hard to study scientists, when you clearly have so much contempt toward them. Also, such a profound misunderstanding of them.

    VA:F [1.9.22_1171]
    Rating: +1 (from 1 vote)
  18. Gomblemomble says:

    I think you must have been reading a different hashtag, from this, to be honest. I have followed #overlyhonestmethods since near the start, and have also contributed. For my own contributions, I can actually say that though one or two are exaggerated for comic effect, some of them are justifications that I have included, worded differently, in published results. For example, I mention the cost of certain chemicals, and the difficulty of obtaining sufficient numbers of some autotrophic bacteria. Both of these are in fact true, and are stated in my published work- because reviewers know more about the specialism than you, they are also aware that these are real, practical limitations on the type of work, and agree that as much as one would ideally like to optimise an experiment, this has to be within achievable parameters and also ones which are repeatable- no use for me to spend years cultivating autotrophs and somehow encouraging them to exceed normal cell density states only for no one else to be able to carry out the same method in future. I’m afraid that if this seems completely alien, you have not really understood empirical science- it is less about doing everything in a particular way, and more about reporting exactly what was done so that it is possible for others to both repeat the method and look at the data obtained with a clear idea of any contributing variables.

    From my reading of the hashtag, I would estimate that about 95% of the tagged items fall within either this area, of the overlap of practical considerations with a desire to carry out ‘the ideal experiment’, or make reference to known human influences within the scientific community which while regrettable, are amusing (for example a friend, who I know to be a good scientist, added something about a supervisor being placed as fourth author on a paper despite not carrying out any research or writing towards it- this is very often the case as sadly scientists are as prone as any other human to prefer work from a known rather than unknown source!) The kind of tweets that ‘reveal’ unacceptable practices are by far a minority, and those I have read (mainly concerned with data manipulation) I believe to be primarily made by very early career researchers (eh, dodgy undergrads is what I mean, really), or to be extremely tongue-in-cheek.

    It actually distresses me that you see it as reasonable to cast doubt over the whole of scientific integrity, and to make veiled threats about future employment/funding based on this meme. If anything, what you should be thinking about when reading silly nonsense alleging data manipulation (if you must interpret it as true, which I do not, and I doubt many do), is that there is a problem in the way science is funded and published. Incentivising ‘positive results’ does, unfortunately, encourage overstatement of conclusions, and perhaps that is what those who have foolishly referred to data manipulation are trying to get at. Funding and publication both are more likely if someone can publish a correlation, rather than saying ‘actually we found no significant link’, which as any scientist who retains some idealism will tell you is also both important and necessary research. I strongly feel that democratising science, and making funding based more on total output rather than ‘publishable output’ or ‘trendy interesting results’ that ‘look cool in Nature’ etc could address this.

    And I’m not going to stop laughing at my fellow scientists being openly silly, either.

    VA:F [1.9.22_1171]
    Rating: +1 (from 1 vote)
  19. #underlyhumoroussociologists

    VA:F [1.9.22_1171]
    Rating: +1 (from 1 vote)
  20. Sandy O. says:

    The bigger problem may be that someone who is “interested in…broader issues related to scientific knowledge and the scientific method” understands so little about scientists.

    VA:F [1.9.22_1171]
    Rating: +1 (from 1 vote)
  21. bill says:

    From another working scientist — what sciliz said, 1000 times.

    The sloppy thinking in this article is why scientists so often view “sociologists of science” with contempt (instead of seeing them as allies, which I think we can all agree is the more productive path).

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  22. Alexander Harvey says:

    From the main piece:

    “This consideration leads me to a related question: how will these confessions be received by the public, who generally speaking, trust that scientific research returns objective results precisely because it adheres to its principles of reliability, replicability and validity?”

    Not being any sort of LabLife, I guess I must be a member of that great unknown.

    I would trust scientific research about as much as I would economics.

    Have you not notice just how little people do trust scientists?

    The age of exceptionalism whether related to a nation, institution, or profession is passing.

    I don’t expect that scientists behave any better or worse than any other lifeforms. Nor do I expect them to do so.

    I read some of #OverlyHonestMethods because is was a hoot. Do I think similar goes on? Yes I do. Do I think it matters? Not really.

    What I do objext to is the notion that we, the masses, are too damn dumb to know the difference between what is an amusing exigency, and malfeasance or cock-up.

    What I really don’t trust are any who believe they carry out their mission statements, individually and jointly, at all times.

    Alex

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  23. Pingback: #OverlyHonestMethods is not an indication of broken science. | Postdoctoral Shenanigans

  24. Ruth Hufbauer says:

    Oh brother! Is what comes to mind reading both the post and the response to the comments. Yes, scientists are human, and the way science works and the ways we learn about the world are informed by the social systems in which we find ourselves. I don’t think that is news to many scientists out there. And yes! We love to crack jokes. While there are, unfortunately, people who cheat, just like in other human endeavors, using a reagent because it comes with a USB stick is a bit like buying salt or flour from the grocery store that gives you a discount on gas or a 1% kick-back on your credit card. It doesn’t affect what you make with that salt or flour. If a scientist discovered that the salt from said store in fact made his bread rise poorly, he or she would go out and try another brand. You are reading _far_ too much into those tweets.

    I think some of the things mentioned might sound shocking for those who simply aren’t familiar with the laboratory procedures that are used, but really, they are things that actually happen but truly play no role in the outcome or are jokes. In rare cases, the ways that scientists get by lead to serendipitous discoveries, but mostly what you’re reading is comic relief.

    I think a few tweets contain serious commentary on the state of science and scientific funding today, and do it in a humorous way. The competition for funding is intense, and while more higher quality science might happen with more smaller grants (somewhat as is done in Canada), the U.S. model these days is to give out fewer larger grants. That is a serious problem, as there are many important questions to be answered that simply don’t happen to be expensive to study. Only a dishonest person (not one likely to joke about it) is going to come up with fancy expensive techniques to answer a question when they are not needed, but it is a real problem that grants tend to be all or nothing.

    I do think that from a sociological perspective there is much to be learned from those tweets, but I don’t think you are on the right track. Me, I was practically crying I was laughing so hard! It was a great 10 minutes out of an otherwise serious day.

    VA:F [1.9.22_1171]
    Rating: +1 (from 1 vote)
  25. Oli says:

    I think this is the first article on the web I’ve ever read where the comments section underneath restores rather than erodes my faith in humanity, thanks to all the articulate, human, intelligent & funny scientists out there, who have to face the realities of silly little technical obstacles and mind-numbing repetitive tasks every day & still deliver first class work. Simon Williams PhD, the joke’s on you – for misunderstanding it. Overly honest research methods are great for science on every level. They provide a snapshot of everyday scientific life that other people don’t normally get to see. They also help stop us scientists from going mad, and we all know what happens when scientists go mad – they get made into professors.

    P.S. Don’t forge your results and if you do, don’t tweet about it. If you do the first of these, you don’t deserve to call yourself a scientist; the second as well & maybe its time to review your claims to be an intelligent life-form.

    VA:F [1.9.22_1171]
    Rating: +1 (from 1 vote)
  26. a says:

    Thank you for providing evidence that certain groups of “scientists” do not develop the necessary ability to know “IRONY”. I could wonder if ethnic background has any influence.
    Hint: when a person is seeing the possibility of doing wrong stuff every day, doesn’t do it, but still uses that as a joke, that’s FUNNY.
    Stupid shows with retarded jokes = NOT FUNNY.
    This is the best humor I’ve seen since Monty Python, so glad and relieved to see it hasn’t DIED.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  27. a says:

    p.s. seeing a science joke these days that doesn’t involve social ineptitude or extreme nerdity is a spark of hope in a very dark world.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  28. Simon Williams PhD says:

    Hi all,

    Thanks for all your comments – I appreciate the feedback and dialogue.

    I wanted to make a couple of quick clarificatory points, as well as place a couple of emphases on my original post. 1000 words is never enough to convey the nuances of such a complex issue in my opinion. And therein lies one of the major problems of the Twitter hashtag – context cannot be conveyed within the confines of 140 characters. As such irony, where it is intended (and justified), MAY be lost on the reader … particularly the uninitiated reader (e.g. one who has not worked in a microbiology lab). That was one point I was making. If the irony and humor has been lost on me in regard SOME cases, then it is indeed because I do not have access to the full range of inside and tacit knowledge, having not worked in a lab. However, following this logic, there is no reason to suggest that, to SOME (note: not all!) uninitiated, non-scientists, the humor MIGHT also be lost. This brings me to my next main clarificatory point:

    The blog sought to ask more questions than it answered. It seems like a number of the responses focus on a couple of the deliberately more provocative questions that I raised along the way in order to stimulate debate. That is fine. However, I wanted to reiterate that really the blog simply asks a series of questions, and in particular, is this good, bad or irrelevant for science, scientists and the scientific method as it is practiced in actuality and as it is presented to the public.

    I see both bad AND a lot of good in this meme. This leads to my next point below. However, irrespective of whether it is good and/or bad in terms of its implications … we can be sure that it does mean SOMETHING. It is interesting sociologically and for the lab community.

    Ok so the final point is clarificatory. As I see it, there are three main types of tweet: 1. those which are fanciful and funny 2. Those which are insightful and interesting 3. Those which allude to the “unscientific”.

    I have elaborated on these three categories elsewhere, and so I wont do at length here – although I did want to clarify for the reponses above that I, like everyone else, do find many of these funny per se … because they are obviously made up just to get a laugh… one does not need to be a bench scientist to see that. Category 1 are the majority of posts, particularly as the meme grows.

    Secondly, there are a large number that are sociologically interesting because they convey the kinds of tacit and insider (socio-cultural) knowledge and practices that sociologists of science used to have to work quite hard to reveal and argue.

    Thirdly, there are a fraction – a relatively small fraction at that – which ALLUDE to the existence of “unscientific” practices. There are two implications: 1. If these in any way truthful, irrespective of whether they are some sort of confession or whether they simply refer to witnessed practices by other people, then these are concerning. They require serious conversation and consideration. If, on the other hand they are entirely made up – and again .. how are we to know or not whether they are via this means? – then they are, at best, irresponsible.

    So, whilst most comments focus on my third point, I wanted to clarify that the issue is far richer than this, and there are interesting questions to be asked.

    The point about tweeting ‘jokes’ about statistical manipulation and massaging or even hiding data was actually intended as a benevolent rather than a belligerent argument. … I personally wouldnt do that, because I dont think it is a good idea to joke so publically about such a serious issue – irrespective of the extent to which this malpractice is pervasive across the sciences or not.

    Finally, I want to acknowledge the comments about the existence of social and economic pressures in this meme. This is also something which has not escaped me… and indeed, the stressful lives of young scientists is something which needs further investigation and support. It is a political commentary in so many ways.

    So, therein lies my conclusion … Im not saying we shouldn laugh at SOME of these … particularly where it provides some light relief from a stressful career … what I am saying is that there is SO much more to this than just light relief … and as well as laughing, we all should also engage in a serious and analytical conversation.

    VN:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  29. Pingback: Creativity and Research, Part I: #overlyhonestmethods | All About Work

  30. Margarita says:

    Gilbert and Mulkay talk about a phenomenon similar to #OverlyHonestMethods in the last chapter (“Joking apart”) of their 1984 book Opening Pandora’s Box: A sociological analysis of scientists’ discourse. There is also a 1982 SSS article by the authors with a similar title. The difference is, of course, that the texts the author’s observe (e.g. ‘do say…don’t say…’ and ‘What he wrote…what he meant…’) are posted *inside* the laboratories and not in the much more public venue of Twitter. Is this significant? Perhaps. If it is, we might ask why now, given that both Twitter and such humor has been around for a while. But my sense is that it has more to do with the the displacement of many previously localized sites of humor to the web than with any confession-like impulse on the part of the scientists. And by extension, I think it reveals more about our new media – rather than scientific – practices.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • Margarita says:

      line 5: authors not author’s*

      VA:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
    • Alexander Harvey says:

      Hi Margarita,

      I think that similar has been around since the start of the internet. People have been share professional humour on BBS and niche but public fora for a long while.

      Some of the humour can be quite dark, my local hospital had a board put up by the nursing staff that had some choice things to say about the inner workings of the hospital. Other groups included software professionals, teachers, airline pilots, tour operators, to my recall. Where they overlapped with things I did know about, much of it feel into the category of hilarious but either true or close to the bone. If it wasn’t edgy it wouldn;t be funny.

      I think the answer to “why now” is that it has been noticed by some that think it must be socially significant, of interest to social science. It has developed nerd appeal.

      I think it is rather healthy behaviour but mostly I think it is outrageously funny.

      I have even known for field anthropologists to send themselves up and point out just how ridiculous and dubious some of their work is, hopefully it that might spread to the rest of the social sciences.

      Alex

      VA:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
  31. Pingback: Bycroft demonstrates how to respond to #overlyhonestmethods | The Bubble Chamber

  32. Pingback: #overlyhonestmethods: You aren’t alone. | GeoMika

  33. Julie says:

    Ugh. Social scientists need not comment. Picking a reagent because it was free? Obviously you don’t work in a lab. There are a million companies selling exactly the same thing. Each one wants you to buy, “theirs” and it doesn’t make a bit of difference which one you choose. Tidy up the spreadsheet? Talk to me when social science creates pages and pages of excel data. If I do the same experiment 45 times, maybe I just label the columns on sheet 1. I could label all 45, but if its in the same order every time do I need to label each page? Especially if I’m the only one seeing the data? This article is a waste of space.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)