> We need something like a Github for scientific data where papers can reference data at a particular commit and then offer improvements through pull requests.
exactly my thought, do you know any working group that is working toward that goal?
exactly my thought, do you know any working group that is working toward that goal?
The basic idea is distributed version control, like git, but over p2p swarms rather than clusters around “central” repositories. We have special handling for large datasets (but still using git) to improve transfer efficiency and diffing.
There’s a UI layer for collaboration (discussion, PRs, review) that supports deep linking to and embedding of files at specific commits, which sounds a bit like what you’re looking for.
Feedback is very much appreciated!
One of the issues for me is file-based versioning, which then requires the means to parse the format. A number of ventures and organizations (e.g., NeuroData without Borders) address versioning of the entire ecosystem necessary to correctly use the underlying data files, so not sure if that’s an explicit part of your ecosystem. Most importantly, is your stack going to be open source?
We support data versioning, interactive web previews, seamless loading into hosted Jupyter notebooks (Kaggle Kernels), seeing/sharing analytic results built on the data version, and adding direct collaborators right now.
We don't support a data-oriented version of an "issue" or a "pull request" quite yet, but these needs are definitely on our radar.