Why is it that comment forums provided by online scholarly journals for post-publication review are so consistently underutilized, even though online forums concerning countless subjects, from culture to politics, are booming? Explanations for this deafening silence in journal comment forums are complex, and reviewed here and here.
My lab’s recent PLOS ONE paper on the bioaccumulation of the antidepressant Zoloft in brewer’s yeast cells was a departure from prevailing wisdom, and so I thought it especially comment-worthy. Determined to buck the trend of no-comments, I proactively solicited comments from other scientists most likely to have informed opinions, reasoning that a personalized email would boost some of them over the commenting threshold, and in turn generate a virtuous cycle of others jumping in.
So I emailed 166 professors worldwide with research interests aligned with mine. Response rates are depicted in the pie charts below. 49 of the 166 were past email correspondents or professional acquaintances (“prior contact”), while the remaining 117 were perfect strangers (“no prior contact”).
Half of the 44 replies were short-lived, terse and congratulatory, e.g., “This looks like an interesting study – thanks for bringing it to my attention! Congrats.” Encouraging, but not informative.
My entreaty was politely declined several times, e.g., “I will not be posting a comment. Sorry – I just don’t do that sort of thing.”
The most negative response was this remark from a full professor with whom I had no prior contact: “Most of us just let our published work speak for itself.”
In the end, I parlayed 22 email exchanges into eight comments, meaning an overall email-to-comment conversion rate of 4.8% (8/166). Of those eight, only two were posted pseudonymously.
This might sound like a poor response rate, but my paper’s comment breakdown was noticeably better than the average profile described in a 2009 analysis, according to which 40% of all PLOS ONE comments were author-generated, and 18% of PLOS ONE papers had at least one comment. Since 2009, however, there has been a torrent of PLOS ONE publications. The most recent statistics indicate that 90% of PLOS ONE articles have zero comments, and that an article with 14 comments (like mine) is in the top 0.001%.
I come away with two conclusions. First, the 90-9-1 online engagement rule basically gets it right. To use a brick-and-mortar analogy, imagine the last conference you attended: for every 100 people who stopped by your poster, 90 said nothing, 9 conversed for a few minutes, but only 1 person became a collaborator. It’s sobering to reflect on comparable ratios applying for online articles, where the convenient option of ongoing asynchronous group conversation exists.
Second, online academic discussion does not come easy, so if you want to cultivate discourse on your paper, be prepared for multiple rounds of gentle prodding to seal the deal. My advice to corresponding authors is simple: activate specific scientific networks, establish rapport and in a handful of cases you’ll be rewarded. Over time, norms can change. I was most encouraged by an associate professor who agreed to move our scientific discussion from emails to a comment thread: “An old dog has learned new tricks and I have posted my comments on your page. I like the idea of an open review process though. I have reviewed a paper for PLOS ONE before but I have never submitted one to them. I may try this out sometime.”
About the Author: Ethan Perlstein is an evolutionary pharmacologist and Lewis-Sigler Fellow at Princeton University. He received his Ph.D. in molecular and cellular biology from Harvard University in 2006, working in the lab of Professor Stuart Schreiber. Ethan’s lab website is http://perlsteinlab.com and he can be followed via Twitter @eperlste.