What I learned as an Academic Editor for PLOS ONE

Open access week is just around the corner, and I thought I’d take the opportunity to share my experience as an Academic Editor for PLOS ONE.

I was invited to join the team following a conversation at Science Online 2010 with I think Steve Koch, who recommended me to PLOS ONE, and before I knew it I was receiving lots of emails asking me to handle a manuscript.

The nice thing about PLOS ONE is that I get to choose which articles I get to handle, and I am very picky. I think that my role is not just to’ handle’ the manuscript but also make sure that the review process is fair. To do this, I need to understand the manuscript myself. I read every article that I take on and write a ‘mini-review’ of it for myself. When I get the external peer reviews I go through every comment they make against the submitted version, compare the different reviews and revisit my first impression of the manuscript. I have learned a lot from the reviewers, they see things I have missed, and they miss things I have detected. It has been a great insight into the peer review process. And I love not having to pull my crystal ball out to determine whether the article is ‘important’ but just having to decide whether it is scientifically solid.

Read/Review

Image by Wiertz Sébastien on Flickr, licenced under CC-BY

If the science is fundamentally good the articles are sent back to the authors for either minor or major changes, and then it falls back into my inbox. I have found it really interesting to see how authors deal with the reviewer’s comments. The re-submission is also a lot of work. I need to compare the original and new version, make sure that the authors have done what they say they have done, make sure that all reviewer’s comments have been addressed. And then I decide if I send it back for re-review or not. One thing that I found interesting in this second phase is when authors respond to the reviewer’s comments in the letter but do not incorporate that into the article. It is almost as if the responses are for my and the reviewer’s benefit only. So back it goes asking them to incorporate that rationale into the actual manuscript. Oh well. That means another round. Luckily this does not happen that often.

And then it is time to ‘accept’ the paper – and so back to the manuscript where I go through commas, colons, paragraphs, spelling mistakes, in text citations, reference lists, formatting, image quality, figure legends, etc. This I normally send to the authors together with their acceptance letter but don’t ask for the article to be re-submitted.

The main challenge I find with the process is time management.

When I get the request to handle an article, I accept or nor based on how much time I have to process the article. That is all good. Except that I cannot predict when the reviews, resubmissions, etc will eventually happen – and many times these articles ‘ready for decision’ show up in my inbox at a time when I cannot give it the full attention it deserves.  Let alone being able to predict when the revised version will be submitted! I find it impossible to plan ahead for this, especially since I have very little control over a lot of my time commitments (like the days I need to lecture, submit exam questions, mark exams). So if an article arrives while I am somewhere at a conference with limited internet connection… How can I plan for this?

Finding reviewers is another challenge. Sometimes they are hard to find. Nothing as discouraging as finding the “reviewer declined…” emails in my inbox indicating that it is back to the system to do something that I thought was done and dusted. The other day someone asked what is a reasonable amount of reviewing one should do a year? My answer was that one should probably at minimum return the number of reviews provided for one’s articles. Say I publish 3 articles a year, each with 3 reviews, then I should not start complaining about reviewing until I have reviewed at least 9 articles. (of course, one can factor in rejection rate, number of authors, etc) but a tit for tat trade-off seems like a fair expectation. So then why is it so hard to find reviewers? Come on people – if it was your paper getting delayed you’d be sending letters to the journal asking how come the article shows as still sitting with the Editor!

And that is the other thing I learned. Editors don’t just sit on papers because they are lazy. There are many reasons why handling an article may take more or less time. In some cases, after receiving the reviews I feel that something has been raised that needs a specialist to look at a specific aspect of the paper. Sometimes I need a second opinion because there is too little agreement between reviewers. Sometimes the reviewers don’t submit in the agreed time. There are many reasons why an article can be delayed, and so what I learned is to be patient with the editors when I send my papers for publication.

But despite the headaches, the stress and the struggle of being an Academic Editor, it is also an extremely rewarding experience. I keep learning more about science because I see a range of articles before they take their final shape, because I get to look into the discussion of what is good and what is weak. And I get to be part of what makes science great: trying to put out the best we can produce.

It is unfortunate that this process is locked up. I think that there is a lot to learn from it. I think that students and early career scientists would really benefit from seeing the process in articles that are not their own, how variable the quality of the reviews are, what dealing well with reviewers comments and suggestions looks like. And the public too would benefit from seeing what this peer review is all about – what the strengths and weaknesses of the process are and what having been peer reviewed really means.

So, back to Open Access week. Access to the final product is really good. Access to the process of peer review can make understanding the literature even better, because it exposes a part of the process of science that is also worth sharing.

 

Creative Commons License
This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution 3.0 Unported License.

This entry was posted in Uncategorized and tagged , , , , . Bookmark the permalink.

8 Responses to What I learned as an Academic Editor for PLOS ONE

  1. Mike Taylor says:

    Editors don’t just sit on papers because they are lazy. There are many reasons why handling an article may take more or less time. In some cases, after receiving the reviews I feel that something has been raised that needs a specialist to look at a specific aspect of the paper. Sometimes I need a second opinion because there is too little agreement between reviewers. Sometimes the reviewers don’t submit in the agreed time. There are many reasons why an article can be delayed, and so what I learned is to be patient with the editors when I send my papers for publication.

    I can accept all that. I just wish editors (or, more sensibly, the automated systems that they use) would communicate more. It would be really helpful, for example, to get an automatic notification saying “one of the referees we solicited to review your manuscript has declined, so we are inviting a replacement”.

    Because one of the most dispiriting parts of the academic process is the sense that our submissions disappear into a black hole for months on end where these mysterious things happen to them. Just demystify. As usual, openness (= more information) is the answer.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • Fabiana Kubke PhD says:

      You make a good point, Mike. I wonder what the best way would be to keep authors informed about what is going on in the background. I am not sure how easy it would be to automate a solution, and I’d be reluctant to pass that responsibility to the editors. I could see, for example, that it might be possible to make it that when you look at ‘status of your manuscript’ it might show you whether reviewers have been invited, accepted, etc. But then again – how would you know that your academic editor is out doing field work with no internet, as I was for several days a couple of weeks ago? Then again, getting closer to the ideal is still better than standing still :)

      VN:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
  2. Karen Daniels says:

    Great piece Fabiana! In response to Mike, one of the things that I’ve really enjoyed about working with PLOS Medicine is that there is less of a black box of automated responses than with other journals. What I appreciate is that the editors actually read what the reviewers say, and add their own responses to this. With some other journals one has the sense that reviews are automatically sent out as soon as they’ve been turned in by the reviewers, without anyone ever having checked what was said. That can be very disheartening when the reviewer is being a little bit “rough”.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
    • Fabiana Kubke PhD says:

      Indeed that is one of the things I like about PLOS. I have been told by more than one person that the editor is to stay ‘removed’. I can see why that might be desirable, but I like getting my hands dirty too. I think I might keep myself more removed was it not for the wonderful community of AEs and PLOS staff that help me navigate the decision-making process when I need it.

      VN:F [1.9.22_1171]
      Rating: 0 (from 0 votes)
  3. Pingback: Building Blogs of Science

  4. JWG says:

    I also serve as an AE for PLoS One (for the past 3 years). In principle, the points made in this article are valid and would appear to be a good method by which one can accomplish a fair yet rigorous review of a ms.
    However, the formal role of the editor is to serve as the judge, whereas the reviewers serve as the jury of peers. In this scenario, having the judge
    behave as a reviewer would be wholly inappropriate. This recently happened at a major journal where the AE insisted on the authors’ doing an experiment to satisfy her/him BEFORE sending the ms out to review. Editors are supposed to serve as dispassionate evaluators of the reviews vs. the claims of the ms. If they serve as both jury and judge, the process gets too muddied. As it is too many reviewers bring their personal politics, likes and dislikes into the review system. The editors are often the last bastion of fairness left in what has become a lamentably flawed review process.

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  5. Fabiana Kubke PhD says:

    Thanks for your comment, JWC. I think I agree with you in principle. But, say for example, you get a paper where it is obvious the controls are missing. You notice this as you go through the paper to choose reviewers. You get the reviews back, and none of the reviewers pointed out to the missing controls, and recommend the paper be accepted as is. You know that having appropriate controls is part of the journal’s policy, and they are simply not there. What do you do?
    Or, you try hard to find reviewers and cant – you only get say 1 reviewer – do you act on that single review or do you give going into more detail over the paper a go?
    As much as I wish that the review process was infallible, it not always is – so as an editor, when you see that the peer review process has failed – what do you do?
    I am really interested in your views, because this is something I struggle with at times.

    VN:F [1.9.22_1171]
    Rating: 0 (from 0 votes)
  6. Dan Woodard says:

    it’s not easy to suggest reviewers (or editors) if you haven’t done this before. Can they be people I know? How do I know if they are interested or feel qualified? If they are at the same institution are they assumed to have a conflict of interest?

    VA:F [1.9.22_1171]
    Rating: 0 (from 0 votes)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>