Hacker News Reader

There is your first mistake: thinking the customer wants to solve the problem in a a good or cost-effective way.

Snowmobile was built because a pre-sales engineer sarcastically said backing up a truck full of hard drives to a customers datacenter would be the fastest way to import X PB of data. It's probably loading from a tape silo and spent 18 months at the customer site.

No, it was built because a solutions architect sat next to a film producer on a plane and talked about how to solve for his unique needs.

Miles, I can't tell if you're messing with me or just stating what actually happened :)

It reads to me like you're saying: if you're going to spend $400k/month on storage cost, why not pay for a network buildout too? I don't think that allocating X per month for op-ex implies that you also have Y amount of money to spend on cap-ex... especially if you don't own the location. 350kw of power is just another trailer to pull up.

I am assuming that anyone who managed to accumulate 100 PB of data (or close enough to make this viable) is still accumulating data at a solid clip, or likely has a lot _more_ than just 100 PB of data

> even relatively remote locations become "accessible" for 100 Gbps+ fiber or microwave connectivity, and _very_ remote locations for tens of Gbps.

I’m guessing:

1. You’ve never priced out said circuits before

2. Never seen the lead times and minimum contract periods on them

3. Don’t realize in general how hard it is to actually max out circuits like that.

Shipping / freight company is generally still a preferred solution to moving very very large amounts of data even in 2020. Your own employer still ships large datasets that way too. Here’s a link from 2007:

https://royal.pingdom.com/fedex-still-faster-than-the-intern...

But from contacts I know inside Google, it’s still done regularly.

Pretty good idea on #3 (and for much much much larger links), and a decent idea for #1.

If I had 1-10 PB to move I could totally see most people using FedEx to ship JBODs around. For small transfers it makes sense, and we also have a product for this. Internally I'd use our WAN for those, which is where this now starts to make sense: Amazon does not have anywhere near the global WAN capacity that we do, nor so far have they had a good reason to have that capacity.

If you've got the global capacity to allow a customer to peer to your nearest location (and your nearest location likely isn't too far away for most of the world), fiber makes sense for large (~EB-scale, which I assume anyone with 100PB+ is well on their way to 1EB+) amounts of data. We have a product for these peerings (primarily for customers in colos): https://cloud.google.com/interconnect/docs/concepts/dedicate...

That product looks exactly like AWS Direct Connect

Also, when I go to: https://www.infrastructure.aws/

(annoyingly I can't link to the network) it says 100Gbps of WAN capacity around the AWS global network.

I did a quick search and wasn't able to find what it is for GCP -- do you have a link handy?

its about timing.

Say I have datacentre in london, Sure I could plumb in 4 100gig lines. But they are typically on a year lease and have a long lead time.

Plus, once its on the datatruck its AWS's problem.

I guess you didn't notice that Google Cloud has a thing that competes with Snowball?

https://cloud.google.com/transfer-appliance

Also, if you want to know an extreme edge case that can't (currently) be done any other way, consider the astronomical observatories at the South Pole.

I said Snowmobile doesn't make sense. Snowball makes total sense. The economics are entirely different.

Antarctica was the only place that came to mind, but I wasn't sure about the costs of getting a shipping container in / out.

For the south pole, they use C-130s, and not standard shipping containers. You can see the little ones that they use in the photo at the bottom of https://www.nsf.gov/geo/opp/support/lc130.jsp

Right, but Snowmobile is packaged as a shipping container because it has a small datacenter inside, so unless you can fit the assembled 45ft container in the plane, I don't think this works?

Those are big fucking planes, so maybe you can?

Obligatory XKCD https://what-if.xkcd.com/31/