In neuroscience, advancements in genetic tools, recording technology, and computer power have paved the avenue to reveal the underlyings of the healthy and diseased brain. Technology development is continuously driving science to new discoveries. We envision the establishment of such a standard and present Exdir to the community as a starting point. Experimental Directory Structure (Exdir) is a proposal to standardize this storage solution. While this strategy can be deployed in various ways by research groups, no common standard for such a storage solution exists. With the publication of Exdir, we invite the scientific community to join the development to create an open specification that will serve as many needs as possible and as a foundation for open access to and exchange of data.Īn alternative storage solution that improves on certain drawbacks of Hierarchical Data Format 5 (HDF5) is to use directories in the file system to define a hierarchy, and store data in binary files, and metadata in text files. Exdir facilitates improved data storage, data sharing, reproducible research, and novel insight from interdisciplinary collaboration. This complicates and limits the opportunity for data sharing and development of common tools for reading, writing, and analyzing data. Several research groups are already using data stored in a directory hierarchy as an alternative to HDF5, but no common standard exists. Exdir uses the same abstractions as HDF5 and is compatible with the HDF5 Abstract Data Model. Exdir is not a file format in itself, but a specification for organizing files in a directory structure.
Furthermore, storing data in multiple files makes it easier to track for version control systems. Exdir, on the other hand, uses file system directories to represent the hierarchy, with metadata stored in human-readable YAML files, datasets stored in binary NumPy files, and raw data stored directly in subdirectories. HDF5 stores data and metadata in a hierarchy within a complex binary file which, among other things, is not human-readable, not optimal for version control systems, and lacks support for easy access to raw data from external applications. We propose a novel alternative, the Experimental Directory Structure (Exdir), an open specification for data storage in experimental pipelines which amends drawbacks associated with HDF5 while retaining its advantages. However, drawbacks related to HDF5's complex specification have initiated a discussion for an improved replacement. Hierarchical Data Format 5 (HDF5) is a popular open format widely used in neuroscience, often as a foundation for other, more specialized formats. Open formats have publicly available specifications which facilitate data sharing and reproducible research. At the same time there is a growing desire to share data along with publications in order to enable reproducible research. Natural sciences generate an increasing amount of data in a wide range of formats developed by different research groups and commercial companies. 5Department of Informatics, University of Oslo, Oslo, Norway.
4Institute of Basic Medical Sciences, University of Oslo, Oslo, Norway.3Department of Biosciences, University of Oslo, Oslo, Norway.2Department of Physics, University of Oslo, Oslo, Norway.1Centre for Integrative Neuroplasticity, University of Oslo, Oslo, Norway.Lepperød 1,4 †, Simen Tennøe 1,5, Marianne Fyhn 1,3, Torkel Hafting 1,4 and Anders Malthe-Sørenssen 1,2 Svenn-Arne Dragly 1,2 * †, Milad Hobbi Mobarhan 1,3 †, Mikkel E.
0 kommentar(er)
