Hacker News Reader

that's horrifying! what if you want to work on the processing algorithms themselves? can't you access the raw input data?

View on HN

If their platform allows any kind of processing as it sounds then anyone could just dump the data. So I don't see why they would not allow downloads for local processing.

Reading between the lines, I assume they aren't reasonably capable of serving up ~TB scale data to hundreds (thousands? more?) of separate clients per day. The usual fix for this would be something like an AWS requester pays bucket.

To your question specifically, in such a scenario they would charge an obscene amount for the bandwidth so you would be strongly incentivized not to download hundreds of GB of data from them.

The questions is, how much are you willing to pay (because the cost of this is not free, especially in this cloud era), and what is the scientific return for access to the uncalibrated data? There is significantly more value in a well designed pipeline for everyone (which is where anyone who would work on the pipeline would slot in) than bespoke pipelines which are poorly built and unmaintainable.

Reply on news.ycombinator.com