AWS Snowmobile – Massive Exabyte-Scale Data Transfer Service | Hacker News Reader

AWS Snowmobile – Massive Exabyte-Scale Data Transfer Service

aws.amazon.com

102 pointsbeef333338 comments

AWS should donate a few of these to the Internet Archive.

They just specifically asked for money to help establish a second copy in Canada.[1]

It would be great publicity for AWS among a demographic not far from their target... AND it would probably benefit AWS itself in some way eventually.

1: https://blog.archive.org/2016/11/29/help-us-keep-the-archive...

EDIT: "a few"?? One trip?

What IS it about AWS and the opaque naming of their services?! There is a reason this exists: https://www.expeditedssl.com/aws-in-plain-english

Those names assume a decent amount of technical understanding. A lot of times the people who will end up paying big bucks for these services don't know what transactional E-mail is, so calling it SES or Amazon Transaction E-mail wouldn't make a difference. So, they go with a "cool" name.

mbeaty_1 above me has a point, but I think Snowmobile & Snowball both tie into the naming for AWS Glacier.

It's such a rarely needed service that I can't imagine naming it AWS Data Center Mover would make any difference at all in terms of usage. The number of customers who could make use of this thing is tiny, it doesn't matter what it is called.

"Never underestimate the bandwidth of a FedEx truck filled with hard disks"

That was an absolutely hilarious "demo." I know it was for real but it was still the sort of thing that makes you just shake your head and laugh. I don't want to be the one paying the S3 bill afterwards though :-)

That latency though..

So disappointed this is not called AWS Station Wagon.

https://en.wikiquote.org/wiki/Andrew_S._Tanenbaum

If one of these jackknifes enroute is that called a 'snowcrash'?

"Ended up in the rhubarb" or similar variation seems like the most appropriate snowmobile slang.

Grandparent was probably referencing the Neal Stephenson novel[0].

[0] https://en.wikipedia.org/wiki/Snow_Crash

I started reading this thinking it was joke, I was very surprised when i realised its a real service. amazing. What kind of applications use so much data and are at the same time for them its cost effective to move them to the cloud?

You're not paying listed rates if you're moving 100PB+ into S3 or Glacier.

Well no, but I'm sure it's still pretty damn pricy.

Oh, no no, I agree with you. Just like Backblaze built their own solution because all other cloud providers were stupid expensive in comparison.

But if you're a CTO or CIO who just wants to shovel the responsibility to someone else, its a solid product at AWS. They're the new "No one got fired for buying X".

Fun, old school massive data transfer as a service. For those of you too lazy to do the math, filling up one of these trucks to capacity (100PB) would run your 500,000 USD per month, ~17k per day (purely for transit).

Edit: I assume that you'd only get charged for the time your vehicle is in transit, though the pricing info doesn't make that super clear. Also, as pointed out, list rates may differ from bulk rates.

Though I assume if you have >100PB to store you aren't paying list rates.

and even if you are (call me, i've got a bridge to sell btw), you should be able to afford it.

It also provides 1Tbps of aggregated bandwidth (in form of multiple 40Gbps interfaces).

According to FAQ, billing starts when it leaves AWS DC and ends after data ingestion is complete.

So roughly minimum (list) price (1 day transit one-way) for transferring that 100PB would roughly be: (100PB/1Tbps * 2 + 2 days) * (($0.005/GB * 100PB)/month) ~= 337k. ~$0.00337/GB

If you are storing that much data, and need to move it, the price tag probably isn't a very big deal for you.

This reminds me of rfc1149 - A Standard for the Transmission of IP Datagrams on Avian Carriers - https://www.ietf.org/rfc/rfc1149.txt

Still, if they made similar terabyte-scale service but using drones, I'd try it just for the heck of it.

Not sure if the writer of that was serious...

Although it was implemented: https://en.wikipedia.org/wiki/IP_over_Avian_Carriers#Real-li...

Will they let you snowmobile your data back out if you want to switch to another cloud provider later? Or is it a one way street, and you pay thru the nose if you want to load your exabytes back out?

The FAQ implies it is for ingest only, and you need to use snowballs if you want to get data out.

As he says in the talk this truck is for 100 Pb, you'd need 10 of them for the aforementioned Exabyte (Eb); which I imagine would nicely equate to a train.

If this data is valuable though wouldn't it need to be securely transported; Mad Max: Fury Road springs to mind...

Increasingly the film Johnny Mnemonic is becoming more and more preposterous; it was 320 Gb in the film. I think the book was 320Mb.

Who even has that much data, outside of Google and Facebook?

I mean - I totally believe there are people that do have that much, I just don't know who they are.

Fake Edit: Film?

There is an example in the blog post about DigitalGlobe.

https://aws.amazon.com/blogs/aws/aws-snowmobile-move-exabyte...

File sharing services. Megaupload back in the day for instance, or services like Dropbox, OneDrive, and Box.

Banks?

That's just numbers and a bit of text, no?

I guess anything media related is way bigger. I thought of media archives of TV stations, but those are (if digital already) cold storage with only a low-res preview live.

Check images, receipts, transactions, statements (particularly PDF), deposit slips, mortgage documents... all kept for 7-20+ years.

Oooh. Very good point.

Any retail chain has that scale of data in surveillance tapes. Order of few mb/s * 10's of cameras per location * 100's of locations * 1000's of seconds per day * 100's of days per year => order of 10^15 bytes per year. So far it hasn't been economical to collect and analyze these tapes. But it is approaching feasibility, and AI is reaching the point it could give useful output on that massive input. Patterns of how shoppers interact with merchandise. What can you change to sell a few percent more items?

" Each Snowmobile consumes about 350 KW of AC power"

Wow.

Maybe Fast and the Furious eleventeen be them hijacking some banks data from AWS snowmobile rigs in transit.

Reply on news.ycombinator.com