Supplementary Information: should I stay or should I go?

On August 11, the Journal of Neuroscience published an Announcement Regarding Supplemental Material by Editor-in-Chief John Maunsell. In it John Maunsell announces that the journal in November will stop accepting supplementary material in article submissions. The announcement has lead to an extensive discussion in the science blogosphere with a number of relevant posts listed below1:

The main arguments against supplementary information are that it overburdens reviewers (and in turn authors), and it counteracts the concept of a self-contained research report. The main argument for supplementary information is that sometimes essential information can’t be provided within the context of a journal article (e.g. video, large datasets), especially in those journals that still have print editions. Several blog posts emphasized that supplementary information is particularly important to provide the research data with the article. I think that Heather Piwowar’s post is the best discussion of the relevant issues.

I’m obviously two weeks late with this blog posts, but I have been on vacation, and I’m only slowly catching up with all the interesting discussions that have happened while I was away. I think the discussion of supplementary information is very important, because it really is a discussion of our concept of a scientific paper. And this concept is changing rapidly for many reasons, including the push to mobile platforms, and the wish by many to publish multimedia files (e.g. 3D structures) and research data with a paper.

Phil Bourne: Beyond the PDF.

It appears to me that we haven’t talked much about supplementary information, and it really has been something everybody was doing out of necessity, without too much thinking about many of the issues, including standard formats, problems for users finding and storing this supplementary information, and copyright issues.

Most journals have (often similar) instructions regarding supplementary information, e.g.:

Supplementary Information is peer-reviewed material directly relevant to the conclusion of a paper that cannot be included in the printed version for reasons of space or medium.
Nature

Although BMC Medicine does not restrict the length and quantity of data in a paper, there may still be occasions where an author wishes to provide data sets, tables, movie files, or other information as additional information.
BMC Medicine

Supplemental Information may include additional control data, validation of methods and reagents, primary data, nonprintable media files, or large data sets. It can also include detailed information regarding Experimental Procedures, including materials (oligonucleotides, plasmids, strains, etc.).
Cell

We strongly encourage authors to include such things as videos, 3-D structures/images, sequence alignments and data sets that are very large, such as those obtained with microarray hybridization experiments.
J Biol Chem

We encourage authors to submit essential supporting files and multimedia files along with their manuscripts. All supporting material will be subject to peer review.
PLoS ONE

Science does not accept as supporting online material HTML files including JavaScript or other scripting languages or Cascading Style Sheets, PowerPoint presentations, and TeX or LaTeX files.
Science

Instead of appearing in the printed version of the journal, supporting information is posted on the PNAS Web site at the time of publication. SI is referred to in the text and cannot be altered by authors after acceptance.
PNAS

The amount of online-only material should be limited and justified. Online-only material should be original and not previously published.
JAMA

The announcement of the Journal of Neuroscience will hopefully initiate a broader discussion of the usefulness and best format of supplementary information. The most interesting aspect for me and several of the bloggers discussing the announcement is the publication of the research data associated with a paper. For now supplementary information is often the only place these data can be published, but there are many reasons why this is not the best idea in the long run.

fn1. This is a perfect example for why we need better systems to track blog posts relating to an article. We have Nature Blogs, Streamosphere or UberVu (and probably others) but they are far from perfect.

Related Posts Plugin for WordPress, Blogger...
This entry was posted in Snippets. Bookmark the permalink.

7 Responses to Supplementary Information: should I stay or should I go?

  1. Maxine Clarke says:

    I mentioned on Stephen Moss’s Nature Network blog post about this decision, that the question of who has responsibility for archiving information is a grey area. I am sure that many editors have sympathy with the J Neuroscience dilemma, and not least referees – all these swathes of material are not only a burden for the referee but often require quite sophisticated software to access, files are huge, etc. I remember the days when referees were only too happy to review Nature submissions because they were short. (Long gone).
    Personally I would like to see institutions and/or funders take explicit responsibility for archiving and maintaining the output of their researchers. A paper is a distillation of some of the work that a person does in a period of time, but there is a lot of other work (negative results, dead ends, repeats that do not get included in the paper, additional controls, etc) which are not strictly “supplementary information” to a scientific paper, but which are nevertheless useful information that should be kept.
    To me it seems logical that all of this should be archived in an institute’s or funder’s repository, as appropriate, and made available to researchers generally at some point. In this way, the information could be built on by others coming later on at that institution or funded by that funder, or accessed by other researchers to inform their own research.

  2. Martin Fenner says:

    Maxine, it’s complicated, and that’s why I hope we have an ongoing discussion about supplementary information. Archiving of research data is just one aspect, and I agree that repositories are a better place than the supplementary information. Initiatives such as “PANGAEA”:http://www.researchinformation.info/products/product_details.php?product_id=260 (which links repository data to published papers in earth system research) are the right direction.
    But what about videos, 3D structures, etc.? Especially for journals such as _Nature_ that also exist as print version? Does it make more sense to publish a paper and supplementary information, or should we rather have two versions of a paper, one of them an enhanced version? The “Article of the Future” format at _Cell_ does something like the latter.

  3. Stephen Curry says:

    As an author I like having the facility of supplementary information since it allows me to include data (usually figures) that cannot easily be accommodated within the printed article.
    As a reviewer, supplementary information can sometime be a huge pain in the ass, especially when there is a lot of it (never mind having to grapple with different file formats). It is useful in at least preventing authors from using the cop-out of ‘data not shown’ which I suspect has been used to mask a multitude of sins in the past. Nevertheless, I would guess supplementary info does not always receive the same level of critical attention as the manuscript.
    I suspect this conversation will continue until there is better integration of data and information sources across the web – we are really only learning this stuff. I agree with Maxine that storage by journal publishers is probably not the best long-term solution. But I wonder if universities or funders are any better? I think I’d like to see more public and more international repositories, such as exist already for some types of data (e.g. the PDB for macromolecular structures – though it is one thing to provide access and quite another to enable newbie users to use the data).
    What about a United Nations of Science to coordinate the activities of scientists around the world? Of course, if we could organise such data repositories in sensible ways, they might start to seem like the obvious places for processing/posting the manuscript itself.
    Solutions will also arise from technology developments – the iPad, the subject of Frank’s recent post, offers huge potential for permitting access to diverse data types within a single device: text, images, video, molecular structure etc. It has yet to be realised but looks like that will only be a matter of time.

  4. Frank Norman says:

    Martin, thanks for this round up. I think this a very I treating area. As you say, it is complex with many stakeholders. As Stephen points out we are only just starting to figure this stuff out.
    I think SI comes about for a number of reasons – because it physically cannot be in the article itself (e.g. Video clips), because there is more data than fits into a paper, because the journal insists on shortening a paper, because reviewers ask for additional data. The format, quality and utility of SI also varies greatly.
    It is therefore hard to generalize about all SI.
    Stephen – I think at big research libraries (University or National libraries, e.g.) are indeed in a good position to carry out archiving roles, and they are already doing this for the humanities.
    Perhaps when we have all the ingredients for effective control – ORCID, Datacite, along with DOIs – it will be easier to devise an architecture for tying together articles, data and whatever else is needed.
    I hope Science Online London will be an opportunity for authors, publishers and service providers to discuss these issues.

  5. Martin Fenner says:

    Stephen and Frank, thanks for your detailed comments. Maybe one good next step would be to think more of what should and should not go into the supplementary information, e.g.:
    *yes*
    Multimedia files (video, etc.)
    *no*
    References, supplemental discussion, supplemental figures and tables
    *no, should go into repository*
    Research datasets
    Once we have repositories for research datasets, and multimedia content in journal articles becomes standard, we might no longer need supplementary information. I’m not so sure about supplementary figures, do we really need them, and where do we put them?

  6. Tom Webb says:

    Martin – thanks for an interesting post on an issue that’s raised quite a lot of interest here at work. Part of the exploding SI problem is surely that we’re being pressured to publish in ‘short paper’ outlets, even when the work we’ve done really requires a much longer paper to fully describe. Trying to squeeze a quart into a pint pot, basically (hence references to ‘supplementary figure S24′, ‘supplementary table S16′, and so on). I pretty much agree with your list of what is or isn’t SI, although I do think it can be a good place for stuff that would traditionally have been in a (print) appendix too (e.g. computer code, mathematical proofs, etc.)

  7. Pingback: Discussing science with microformats | Gobbledygook