New in PLoS ONE: Citation rates of self-selected vs. mandated Open Access

PLoS ONE today published a paper very relevant to Open Access Week (which started today):

Gargouri Y, Hajjem C, Larivière V, Gingras Y, Carr L, Brody T, Harnad S. Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research. PLoS ONE. 2010;5(10):e13636+. doi:10.1371/journal.pone.0013636.

The paper studied the citation rates of papers from four institutions with the longest-standing self-archiving mandate: Southampton University, CERN, Queensland University of Technology and Minho University. The mandates (instituted between 2002 and 2004) increased the rate of self-archiving in the respective institutional repositories from around 15% to 60%:

Fig 1. Open Access Self-archiving percentages for institutions with self-archiving mandates compared to non-mandated, self-selected controls. doi:10.1371/journal.pone.00136363.g001

The authors then compared the citation rates of self-archived papers from the four institutions published in non-Open Access (OA) journals (mandated OA) to matched papers from the same non-OA journal, volume and year (non-OA). 15% of the latter papers were also available from repositories (self-selected OA). Overall the study examined the citations of 27,197 papers from 1,984 non-OA journals. The study also looked at the influence of other factors that affect citation rates, e.g. article age, journal impact factor or number of references.

The main results of the study are that OA articles are cited more frequently than non-OA articles and that there is no difference in citation rates between mandated and self-archived articles. The authors conclude that OA articles are cited more often not because of self-selection (articles with higher impact more likely become OA), but because of readers cite OA articles more often.

The study is an effort to better understand if and why OA papers are cited more often, the so-called OA Advantage. Unfortunately I feel that the paper comes a little short. Yes, they did a very detailed analysis of the citation behavior, and take into account important cofactors. But the reader is left with the impression that mandatory self-archiving of post-prints in institutional repositories is the only reasonable Open Access strategy, and the introduction and discussion accordingly leave out some important arguments.

The authors fail to mention that not all studies show a citation advantage for OA articles (e.g. the randomized controlled trial by Davis PM et al. 2008), and they only look at self-archiving in institutional repositories (green OA). Not only are there important disclipline-specific repositories such as PubMed Central and ArXiv, but of course also OA journals such as PLoS ONE that published the study (gold OA). And the study only discussed archiving of articles after peer review, even though archiving of preprints is a popular strategy in high-energy physics and related disciplines (and here the self-archiving rate is close to 100%). I also don’t see a discussion of why the self-archiving rate of post-prints in institutions with a mandate is 60% and not close to 100%. And what is wrong if archiving rates are only around 15% if it is left to the researchers to decide? Finally, there are many reasons other than citation rates that make OA worthwhile, including access to the literature for those not working in academic institutions (and maybe never citing a paper), and work that uses the OA literature in new and exciting ways that go beyond citations. For me personally, the OA advantage is much more in these two aspects than in the citation rates of papers.

For me Open Access week is an opportunity to celebrate what has been achieved over the years. But it is equally important to look at the road ahead. There is no “one size fits all” solution for Open Access, and we shouldn’t look away from the things that aren’t working, and the solutions that haven’t been found yet.

Gargouri, Y., Hajjem, C., Larivière, V., Gingras, Y., Carr, L., Brody, T., & Harnad, S. (2010). Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research PLoS ONE, 5 (10) DOI: 10.1371/journal.pone.0013636

Related Posts Plugin for WordPress, Blogger...
This entry was posted in Conferences, Interviews, Presentations, Recipes, ResearchBlogging, Reviews, Snippets, Thoughts and tagged . Bookmark the permalink.

12 Responses to New in PLoS ONE: Citation rates of self-selected vs. mandated Open Access

  1. SOME METHODOLOGICAL SUPPLEMENTS

    (1) Yes, we cited the Davis et al study. That study does not show that the OA citation advantage is a result of self-selection bias. It simply shows (as many other studies have noted) that no OA advantage at all (whether randomized or self-selected) is detectable only a year after publication, especially in a small sample. It’s since been over two years and we’re still waiting to hear whether Davis et al’s randomized sample still has no OA advantage while a self-selected control sample from the same journals and year does. That would be the way to show what the OA advantage is a self-selection bias. Otherwise it’s just the sound of one hand clapping.

    Harnad, S (2008) Davis et al’s 1-year Study of Self-Selection Bias: No Self-Archiving Control, No OA Effect, No Conclusion. Open Access Archivangelism. July 31 2008.

    (2) No, we did not look only at self-archiving in institutional repositories. Our matched-control sample of self-selected self-archived articles came from institutional repositories, central repositories, and authors’ websites. (All of that is “Green OA.”) It was only the mandated sample that was exclusively from institutional repositories. (Someone else may wish to replicate our study using funder-mandated self-archiving in central repositories. The results are likely to be much the same, but the design and analysis would be rather more complicated.)

    (4) Yes, we systematically excluded articles in Gold OA journals from our sample, not because we do not believe that they generate the OA advantage too, but because it is impossible to do matched-control comparisons between OA and non-OA articles in the same journal issue with Gold OA journals, since all their articles are OA. (It would for much the same reason be difficult to do this comparison in a field where 100% of the articles were OA, even if we were interested in unrefereed preprints; but we were not: we were interested in open access to refereed journal articles.)

    (5) As to mandated self-archiving rates: The institutions we studied had mandated OA in 2003-2004. Our test time-span was 2002-2006. At least two of those institutions (Southampton ECS and CERN) and probably the other two also (Minho and QUT) have deposit rates of close to 100% by now. (We have since extended the analyses to 2009 and found exactly the same result.)

    (6) “What is wrong if [OA] rates are 15%”? We leave that to the reader as an exercise. That, after all, is what OA is all about. But surveys have shown — and outcome studies have confirmed — that although most researchers do not self-archive spontaneously, 95% report that they would self-archive if their institutions or funders required it, over 80% of them saying they would do it willingly. (Most don’t self-archive spontaneously because of worries — groundless worries — that it might be illegal or might entail a lot of work.)

    (7) Yes, “there are many reasons other than citation rates that make OA worthwhile,” but if most researchers will only provide OA if it is mandated, then it is important to demonstrate to researchers why it is worth their while.

    (8) If we have given “the impression that mandatory self-archiving of post-prints in institutional repositories is the only reasonable Open Access strategy,” then we have succeeded in conveying the implication of our findings.

  2. Martin Fenner says:

    Stevan, congratulations for publishing the paper, and for all the hard work that went into it. And thanks a lot for the clarifications. I do believe that it is important to continue to study why self-archiving is not popular unless mandated, especially since I come from a country (Germany) where we have reservations against mandates for researchers because of our historical past (1933-45). And what you say in (8) nicely summarizes where our thoughts about Open Access differ.

  3. But Martin, one must not put too much weight on choice of words! If you don’t like the word “mandate,” pick the world “requirement,” “regulation” or “rule.” There is nothing wrong with useful, constructive, beneficial rules, is there? And Germany (like all countries, and all institutions) has, and must have, rules! Indeed, one rule that Germany shares with the global academic world is “publish or perish,” which means that if you are a researcher, you are expected to conduct research and report your findings (otherwise you will not be hired or promoted or funded).

    “Publish or perish” is already a universal academic “mandate,” even if it is not usually referred to as such. Well, an open access mandate is merely a slight, natural extension of publish-or-perish for our new, PostGutenberg online era: publish-or-perish and self-archive-to-flourish.

    Already universities are not merely counting publications, like beans, in rewarding research performance. They are also counting how much the publications are used, how much they are taken up and built upon by other research. How much an impact they are having on research progress. An indicator of this research impact is citations. And it is exactly citations that our PLOS ONE study showed that OA increases.

    Remember that we are talking here about refereed research journal articles, not about books, textbooks, video, audio, software: Research articles are all author-giveaways, written only for usage and impact, not for earning royalties. (Most of the rest of digital content is not.)

    In Germany — for some reason I really cannot fathom! — OA mandates have been persistently misperceived and misrepresented as being some sort of infringement on academic freedom. But no one is infringing on academic freedom. Authors continue to publish what and where they choose. In addition, however, they are required to make their (give-away) publications accessible not only to those users whose institutions can afford subscription access, but to all potential users — by depositing their final, refereed drafts in their institutional repositories…

  4. Martin Fenner says:

    Stevan, I think that every country has proponents and opponents of Open Access. We in Germany had opposition to OA in the form of the Heidelberger Appell in 2009, the United States had PRISM in 2007. The major German funding organizations have a very clear position on Open Access, and in fact they all signed the Berlin Declaration in October 2003.

    Let me get back to mandates for self-archiving. Would we need mandates if the advantages for authors – including the OA Advantage for number of citations that your paper demonstrated – were large and obvious? I think the OA advantages for readers (free access and Creative Commons license that allows reuse) are much greater than for authors (who might even have extra costs and work with self-archiving). But is of course the author who decides where to publish and whether to self-archive. The argument for a mandate might be that the value for readers exceeds the burden for authors.

    But I would rather work on incentives for authors – I personally don’t think that the citation advantage discussion of the past few years has convinced many authors. Like you I think that new research metrics that go beyond citation rates have the potential to show a clear OA advantage. Cameron Neylon is working on an interesting proposal in that direction.

  5. ZENO’S PARALYSIS AND ITS CURE

    Only 15% of authors self-archive spontaneously because more authors are either worried that self-archiving is (1) illegal, (2) prejudicial to their chances of getting published, (3) a lot of work or (4) will destroy publishing. All four worries (and 34 others) are completely groundless, and one can explain that toresearchers one by one, but that would take till the heat death of the universe, because 85% of all researchers is a lot of researchers.

    And, yes, they want greater research impact, but telling them OA will provide it is not enough to convince them either. In Alma Swan’s international, cross-disciplinary surveys they make it clear that they will only provide OA if and when their institutions or funders mandate it.

    The solution is hence obvious. (Maybe this year’s OA Week OA Mandate Challenge will get the message through to the provosts, VCs and Rectors that that is the solution. Please see EnablingOpenScholarship and EOS for the details.)

    http://www.openscholarship.org/jcms/c_6226/open-access-policies-for-universities-and-research-institutions?hlText=policie

    http://www.openoasis.org/index.php?option=com_content&view=article&id=148:the-optimal-open-access-policy-for-institutions&catid=81:articles&Itemid=340

    Harnad, S. (2006) Opening Access by Overcoming Zeno’s Paralysis, in Jacobs, N., Eds. Open Access: Key Strategic, Technical and Economic Aspects, chapter 8. Chandos. http://eprints.ecs.soton.ac.uk/12094/

    Swan, A. (2006) The culture of Open Access: researchers’ views and responses. In : Jacobs, N., Eds. Open Access: Key Strategic, Technical and Economic Aspects. Oxford : Chandos/ 52-59, 2006 http://eprints.ecs.soton.ac.uk/12428

  6. Pingback: Quick Links | A Blog Around The Clock

  7. Martin Fenner says:

    Stevan, again thank you for providing good arguments for self-archiving. I will try to look at this in more detail by writing a separate blog post, something like chances and fears of self-archiving from a researcher perspective.

  8. Pingback: Open Access News 10/23/2010 |

  9. Pingback: Metodi scientifici » Ocasapiens - Blog - Repubblica.it

  10. Pingback: Self-motivated vs. mandated archiving | Gobbledygook

  11. Pingback: Settimana dell’Open Access « Oggi Scienza

  12. Pingback: Echo Chamber