Hacker News Reader

Comment by kypro | Hacker News Reader

From the comments:

> Time for a data transfer startup based on pre-programmed drones with relay terminals to recharge or change carriers.

I know it's a joke, but I'd be interested in knowing if anyone knows of any theoretical or known commercial use cases of exactly this?

If you're a corporate with a petabyte of data you want to send from the US to Europe to do some data analytics on, would you not back it up to tape and load it on a plane? Seems like this must happen from time to time, no?

View on HN

For terabyte-scale data it might make sense to send it along with a person, but for larger amounts there are definitely services to help handle the process.

He mentioned AWS Snowmobile in the video, but they also have a lower-volume equivalent: AWS Snowball [1]. It's basically just a suitcase-sized device that you plug in, upload data to, and ship to the destination. They have some really nice features to help make the process smooth. But yeah, at its core the idea really is "pack up a lot of hard drives and have it shipped to the destination".

1: https://aws.amazon.com/snowball/

Also reminds me of the XKCD What-if: "When - if ever - will the bandwidth of the Internet surpass that of FedEx"

https://what-if.xkcd.com/31/

What-ifs really need to be dated. Some of those assumptions are enough to date it; 160tb/kg is kind of laughable now, when talking about micro SD cards.

Reasonably guessable using https://colin-scott.github.io/personal_website/research/inte...

"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway." –Andrew Tanenbaum, 1981

Nowadays you can buy 30TB 2.5 inch SSDs which includes a durable metal housing. So a station wagon could feasibly transport close to 300 petabytes. Though that would be close to a quarter billion dollars worth of SSDs.

Are ssd the best alternatives regarding price/bite and volume/bite ? It seems and SDUC card will largely beat on the volume comparison.

Not if you expect to actually get data out of them after hurtling down the road. Even for top end enterprise grade SSDs made at the highest standard, I would only assign a <99.9% chance of doing so. So that's 10 bad SSDs per trip at least.

Erasure codes are your friend, if you have the compute.

it's not commercially available yet, but I imagine 5d storage would be best data density.

This isn't very uncommon in geospatial companies, where data is often hundreds of terabytes. I remember there was open government data you could request from USGS (high res satellite imagery of all of the US), it was technically free, but you couldn't download it. You requested it, filled out a form, and they shipped you 4-5 drives with bubblewrap.

I suppose it mostly depends on how much the saved time is worth and how that compares to the cost of transport. I suppose there are some cases where shipping a box of drives makes sense.

I think a much more common case is having a critical machine crash and needing a backup sent to you. Depending on the bandwidth you actually have, restore by mail can be faster. And that's why companies like backblaze offer it.

https://www.backblaze.com/cloud-backup/features/restore

Restoring 8TB, which is not an unreasonable amount of data in 2023, at 40Mb/s, which is not that uncommon in much of America, still takes more than 2 days when everything goes perfectly.

FedEx'ing a 8TB drive should be able to beat that in many cases.

I introduce to you https://aws.amazon.com/snowmobile/

Use case, really interesting:

https://blog.maxar.com/earth-intelligence/2017/digitalglobe-...

In the genomics field this is why we are working on making “data visiting” easier: https://www.ga4gh.org/product/workflow-execution-service-wes...

Ditto in my previous job, there were weekly package deliveries of hard drives from Europe, which were a magnitude faster (then) compared to what was possible for a reasonable price... Of course this was all back in the day, and might only be economical these days from somewhere with major airports but no connecting fibre.

At my previous job we had to get data off a ship after a test. The upload rate off the ship was slow. The solution was to put the data on drives, put them in a helicopter, and fly them to land.

Not quite the same, but Peter Jackson famously used iPods to transport dailies while filming LoTR.

i can't find a source for it, but i know nasa has done this in the past.

Reply on news.ycombinator.com