All-or-Nothing versus Partial Peer Review, by PractiCal fMRI

By PractiCal fMRI

One of the many aspects of post-publication peer review (PPPR) I enjoy is an ability to provide feedback on just a small component of a paper. More often than not I will critique an acquisition method weakness, since MRI acquisition is my forte. This selectivity has made me wonder why we don’t have a similar fractional reviewing option for the pre-publication peer review process. It would allow me to accept (partial) reviews of many more manuscripts sent my way.

Let’s deconstruct this a little bit, see what we’re dealing with. I’m going to stick with functional MRI because, as mentioned, some part of it is my forte. But fMRI may be an instructive example for any massively interdisciplinary field. To interpret fully a manuscript describing an fMRI study will require a reviewer possess considerable expertise in acquisition methods involving physics and physiology, image processing, experimental design, statistics and neuroanatomy, plus the authors’ stated academic field of endeavor. Let’s say that the manuscript under consideration is an fMRI study of language comprehension, so language is the field of endeavor. Let me try to think of five people I consider savvy with all of the first five categories I listed. Yep, I can think of a small handful of people. None of them is an expert in language comprehension, however. Now let me think through all the people I know researching language. None of them is an expert in more than one or two of the first five categories.

I assume that journal editors are all too aware of the problem of finding two or three individuals capable of providing a comprehensive, expert review of every component of a manuscript on fMRI. Presumably they already sub-divide the review so that one person will be able to handle, say, the acquisition and image processing, another will be able to handle the stats and experimental design, and perhaps a third will be expert in language comprehension, to continue with the worked example.

If, however, the reviewers are all biased in one direction – my bet is that language comprehension would be the most common bias because potential reviewers are likely to decline invitations to review work they’re not fundamentally interested in – then we could easily see cursory reviews of the other parts. Like, say, the acquisition or the statistics. Sounding familiar yet?

We often hear (or express) the refrain, “I can’t believe that got past the reviewers!” How about we encourage journals to adopt partial peer review as an option to total review? By all means give me the option to review a manuscript in toto, but if I decline why not ask me if I would be willing to look at part of it? Provided I can delineate the pieces I have reviewed then the editor can set about assigning reviewers to the remaining parts.

I recognize that partial reviews may make the editor’s job more involved. On the other hand, it may mean an increase in the potential number of reviewers as well as their level of enthusiasm to review, netting a benefit that might actually lower the aggregate workload if we assume that there is a lot of editorial legwork involved in getting rejection after rejection and then having to chase reviewers to get their (full) reviews conducted in a timely manner.

Right now I tend to decline about two thirds of manuscripts I’m sent. Infrequently it is because I am too busy. (Hey, I’m being honest!) Mostly it is because, having read the abstract, I suspect that I will be expert in a small fraction, passably competent in another fraction and out of my depth for enough of it to render the overall review flawed. If I am sufficiently interested in the work to want to try a review regardless, it means that I shall need to get a working knowledge of the parts I’m weakest on and hope for the best. If I ask a colleague for assistance then, surely, what I have just created is de facto partial peer review.

But would formal partial peer review work in a pre-publication model? Parts of certain manuscripts are certainly good candidates for expert outsourcing: the methods and the statistics in fMRI papers, for example. Would that put an excessive burden on those cursed with competence in these domains? Would they resent being “used” only for their expertise in a restricted domain, like a technician? And would it then prove difficult for an editor to piece together the fractional reviews to ensure a whole? What do you think? From now on we will have PPPR to conduct partial peer review whenever we like, but if there is a way to make even a small improvement in the quality of pre-publication reviews, shouldn’t we explore it?

practiCal fMRI is a physicist and facility manager who has been doing MRI since 1988. He has worked at fields as high as 17.6 T and as low as 130 uT. He runs an educational blog in lieu of writing a text book, at practicalfmri.blogspot.com and is @practicalfmri on Twitter.