Hacker News Reader

Comment by ThisIsTheWay | Hacker News Reader

For a sense of scale, my company used an AWS Snowmobile to transfer 100PB of archived satellite imagery to AWS.

More details here: https://blog.maxar.com/earth-intelligence/2017/digitalglobe-...

For a sense of the world, I went to Manila in 2018 to find out why there are several distributor can only sent their data weekly.

One of the distributor show me a 64 MB usb flashdisk that’s travelled every week with the delivery truck to get 4 MB of data from their branches (8 hours truck driving from the capital)

Went to Myanmar in 2017. The nation is just upgrading to 4G in 2016. But not every places have 4G coverage

2019 in eastern Indonesia (Seram island) you’re lucky if you can get a reliable 5 KB/s internet. It’s a remote area, but we still have a branch there :(

Why not satellite internet then?

$$$

"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."

Or these days, micro flash cards.

For real! Reminds me of this article...

"A carrier pigeon with a cargo of microSD cards can transfer large amounts of data faster, and more cheaply, than just about any other method"

Link: https://spectrum.ieee.org/tech-talk/computing/networks/pigeo...

This almost got me and then I looked at the date of publication.

If you're new to joke RFCs, I suggest reading up on RFC 3514, a suggestion that malicious IP packets should be required to set the high order bit of the IP fragment offset field, which is to be designated the "evil" bit. This significantly simplifies implementing effective network monitoring.

It's still true. A pigeon can easily carry 10 microSD cards, 1TB each. Even if you're on a gigabit connection, it will take you over 22 hours to copy 10TB.

They meant that it was posted on April fools, not that it's an old article.

What was the premium of storing the data in S3 as opposed to keeping it in your own tape warehouse? Definitely curious about the cost benefits!

Unfortunately I don't have access to that info, sorry!

Is your tape library stored redundantly across multiple data centers?

S3 Glacier Deep Archive is probably the closest to a tape warehouse, at the standard rate it would cost $99000/month to store 100PB of data.

$0.00099 per GB per month * 100,000,000 GB = $99000

You need to account data transfer OUT if you need to get your data from S3. _If_ you could download 100PB data, it'd cost you $1M.

Hopefully it is not stored as one giant object and hopefully your retrieval would be only specific parts (small parts.)

You’d want to do some research on getting data out of glacier. I’ve read some nightmare stories about the bill from trying to get a small amount of data out of glacier. I can’t remember the details, and some of the large cost was people sort of doing it wrong. But some of the tooling makes it easy to do wrong.

When I was using glacier, we treated it sort of like a write only storage, with the understanding that if we ever needed data out, we’d have to think long and hard about the data’s value, and probably would have gotten an AWS rep on the line to make sure we didn’t do anything stupid.

Just FYI your company has a cert mismatch on the following link from that blog post: https://platform.digitalglobe.com/gbdx/

The cert is limited to *.wpengine.com

And the link 404s so.. that's a shame. Was interested in learning more about the platform.

Thanks for the heads up! I'll notify the team.

You can learn more here: https://www.digitalglobe.com/products/gbdx

Like everything from DigitalGlobe -- it doesn't work.

LOL. What products have you had bad experiences with?

That was an excellent read. Thanks for sharing.

Reply on news.ycombinator.com