MESSAGE
| DATE | 2017-06-02 |
| FROM | Ruben Safir
|
| SUBJECT | Subject: [Hangout - NYLXS] Neural Sciences and Coding
|
http://www.nature.com/neuro/journal/v20/n6/full/nn.4550.html
Toward standard practices for sharing computer code and programs in
neuroscience
Stephen J Eglen1, Ben Marwick2, Yaroslav O Halchenko3, Michael
Hanke4, 5, Shoaib Sufi6, Padraig Gleeson7, R Angus Silver7, Andrew P
Davison8, Linda Lanyon9, Mathew Abrams9, Thomas Wachtler10, David J
Willshaw11, Christophe Pouzat12, Jean-Baptiste Poline13,
Affiliations
Corresponding authors
Nature Neuroscience
20,
770–773
(2017)
doi:10.1038/nn.4550
Published online
25 May 2017
Article tools
PDF
PDF
Download as PDF (618 KB)
View interactive PDF in ReadCube
Citation
Reprints
Rights & permissions
Article metrics
Computational techniques are central in many areas of neuroscience and
are relatively easy to share. This paper describes why computer programs
underlying scientific publications should be shared and lists simple
steps for sharing. Together with ongoing efforts in data sharing, this
should aid reproducibility of research.
Subject terms:
Neuroscience
Scientific community
Many areas of neuroscience are now critically dependent on computational
tools to help understand the large volumes of data being created.
Furthermore, computer models are increasingly being used to help predict
and understand the function of the nervous system. Many of these
computations are complex and usually cannot be concisely reported in the
methods section of a scientific article. In a few areas there are widely
used software packages for analysis (for example, SPM, FSL, AFNI,
FreeSurfer and Civet in neuroimaging) or simulation (for example,
NEURON, NEST, Brian). However, we often write new computer programs to
solve specific problems in the course of our research. Some of these
programs may be relatively small scripts that help analyze all of our
data, and these rarely get described in papers. As authors, how best can
we maximize the chances that other scientists can reproduce our
computations, find errors or reuse our methods on their data? Is our
research reproducible1?
To date, the sharing of computer programs underlying neuroscience
research has been the exception (see below for some examples) rather
than the rule. However, there are many potential benefits to sharing
these programs, including increased understanding and reuse of your
work. Furthermore, open source programs can be scrutinized and improved,
whereas the functioning of closed source programs remains forever
unclear2. Funding agencies, research institutes and publishers are all
gradually developing policies to reduce the withholding of computer
programs relating to research3. The Nature family of journals has
published opinion pieces in favor of sharing whatever code is available,
in whatever form4, 5. Since October 2014, all Nature journals require
papers to include a statement declaring whether the programs underlying
central results in a paper are available. In April 2015, Nature
Biotechnology offered recommendations for providing code with papers and
began asking referees to give feedback on their ability to test code
that accompanies submitted manuscripts6. In July 2015, F1000Research
stated that “software papers describing non-open software, code and/or
web tools will be rejected”7. Also in July 2015, BioMed Central
introduced a minimum-standards-of-reporting checklist for BMC
Neuroscience and several other journals, requiring submissions to
include a code availability statement and for code to be cited using a
DOI or similar unique identifier8. We believe that all journals should
adopt policies that strongly encourage or even mandate the sharing of
software relating to journal publications, as this is the only practical
way to check the validity of the work.
What should be shared?
It may not be obvious what to share, especially for complex projects
with many collaborators. As advocated by Claerbout9 and Donoho10, for
computational sciences, the scholarship is not the article; the
“scholarship is the complete software [...]”10. So, ideally, we should
share all code and data needed to allow others to reproduce our work,
but this may not be possible or practical. However, it is expected that
the key parts of the work should be shared, for example, implementations
of novel algorithms or analyses. At a minimum, we suggest following the
recommendation of submission of work to ModelDB11, i.e., to share enough
code, data and documentation to allow at least one key figure from your
manuscript to be reproduced. However, by adopting appropriate software
tools, as described in the next section, it is now relatively
straightforward to share the materials required to regenerate all
figures and tables. Code that already exists, is well tested and
documented, and is reused in the analysis should be cited. Ideally, all
other code should be communicated, including code that performs simple
preprocessing or statistical tests and code that deals with local
computing issues such as hardware and software configurations. While
this code may not be reusable, it will help others understand how
analyses are performed, find potential mistakes and aid reproducibility.
Finally, if the work is computationally intensive and requires a long
time to run (for example, many weeks), one may prefer to provide a small
'toy' example to demonstrate the code.
By getting into the habit of sharing as much as possible, not only do we
help others who wish to reproduce our work (which is a basic tenet of
the scientific method), we will be helping other members of our
laboratory or even ourselves in the future. By sharing our code
publicly, we are more likely to write higher-quality code12, and we will
know where to find it after we have moved on from the project13, rather
than having the code disappear on a colleague's laptop when they leave
your group or suffer some misfortune14. We also will be part of a
community and benefit from the code shared by others, thus reducing
software development time for ourselves and others.
Simple steps to help you share code
Once you have decided what to share, here are some simple guidelines for
how to share the work. Ideally, these principles should be followed
throughout the lifetime of the research project, not just at the end
when we wish to publish our results. Guidelines similar to these have
been proposed in many areas of science15, 16, 17, suggesting that they
are part of norms that are emerging across disciplines. In the 'Further
reading' section (Box 1), we list some specific proposals from other
fields that expand on the guidelines we suggest here. Box 2 describes
several online communities for discussing issues around code sharing.
--
So many immigrant groups have swept through our town
that Brooklyn, like Atlantis, reaches mythological
proportions in the mind of the world - RI Safir 1998
http://www.mrbrklyn.com
DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002
http://www.nylxs.com - Leadership Development in Free Software
http://www2.mrbrklyn.com/resources - Unpublished Archive
http://www.coinhangout.com - coins!
http://www.brooklyn-living.com
Being so tracked is for FARM ANIMALS and and extermination camps,
but incompatible with living as a free human being. -RI Safir 2013
_______________________________________________
Hangout mailing list
Hangout-at-nylxs.com
http://www.nylxs.com/mailman/listinfo/hangout
|
|
