Hacker News Reader

Comment by Anon84 | Hacker News Reader

Many times this is completely impossible. I still have an email from the sysadmin of (at the time) a top 500 supercomputer asking of me and the people I was working with could please delete some of the 210TB we generated for a specific project that eventually resulted in 4 or 5 papers.

View on HN

Your data is still digital at least. What about the E. coli evolution experiment (http://en.wikipedia.org/wiki/Escherichia_coli_long-term_evol...) that has to store its data in analog form in a freezer? You can't send that to other people with interest in the results by e-mail or FTP.

That sounds expensive, not impossible in principle.

The past and current community standard is that is not absolutely necessary to share data, so people run projects for which it is prohibitively expensive to copy the data.

If we chose to weight sharing more highly, we could budget for sharing and/or scale project data to make it more sharable. We might decide that if the data are too expensive to share in practice, we don't fund the project. We choose. It's not impossible.

The argument is that you should be able to regenerate the data from scratch, assuming it is a simulation of some kind. This just requires source code and config files, which are still generally not shared.

One of the problems with that was that some of the input data we were using was commercial (and almost as expensive as a postdoc years salary).

The best we could come up with was to publish as many details about the simulations as possible and let people run a limited number of simulations on our own servers (see www.gleamviz.org).

Edit: Even so you can't actually redo the original simulations as there have been several updates since, including a complete rewrite of the simulator, updated input databases, etc...

Maybe researchers could negotiate a license for commercial data that would allow others access to the data for review or reproduction purposes at a reasonable charge, but not for use as the basis of new claims.

Has anyone seen a deal like this before?

It would never happen, people would steal/share the data. Remember researchers (specifically in academia) are typically short on cash, and will cut corners whenever possible. The good ones (a majority in my experience) won't cut corners when it comes to the integrity of their data though.

But you can't cite stolen data, and you can't base claims on things you don't cite. You would pay at least when it came to submission time.

Reply on news.ycombinator.com