Bypass the internet for large transfers: AWS Import/Export | Hacker News Reader

Bypass the internet for large transfers: AWS Import/Export

aws.amazon.com

86 pointsmattjaynes30 comments

About 3 months ago, I needed >50Gb of data uploaded to AWS and I asked a friend of mine who works at Amazon for help. He asked around and told me to mail the external storage to Amazon with keys/bucket info and a special code. I UPSed the stuff and within 2 days of arrival, I received an email with a notification that the transfer was done. Later on, I got the storage back. Everything worked as advertised!

I guess there's so much demand for this sneakernet that they made an official service. I wonder how many other people asked for transfer help before today's announcement...

If it was only three months ago, more likely this service was already in development and you were customer #1 without knowing it.

Unlikely. I have 2 contacts that did the same thing months ago. I actually thought that this was an already advertised service.

When I read the announcement this morning, it occurred to me that the reason I like AWS so much, and what separates them from other cloud offerings, is that they're doing stuff dev teams actually want and can use (not impenetrably complex tech we could use with some vendor consultancy). Import features like this are trivial for AWS to implement - but will be a step change for some customers.

Last generation vendors saying 'anyone can do what AWS are doing' is just crap. You can't buy the enthusiasm these guys have - and its not just one individual - if you talk to anyone from AWS (and they're often around nights/weekends to talk about stuff) they are in a different lane to the competition or vendors that should be the competition.

Agreed, except I have found it pretty tough to figure out EC2. EC2 is fairly impenetrable at first.

Obligatory Tanenbaum quote:

"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."

Indeed, so obligatory that I included it in my blog post:

http://aws.typepad.com/aws/2009/05/send-us-that-data.html

Kudos (to all of you at Amazon) for being the first to turn that quote into a product feature. Certainly seems obvious in retrospect.

On the contrary, video delivery networks (VDNs) have been recommending loading data this way for years, such as one that's routinely internationally FedEx-ing those cheap 2TB "network" drives back and forth in small white mil-spec shipping cases.

Sometimes a big truck has more bandwidth than a series of tubes...

Bandwidth and latency are two different things. But it seems we are so often confused by these two concepts.

I feel funny because Werner Vogels is Andy Tannenbaum's phd student. That's how a concept propagates from one generation to next.

Terrible latency, though.

Living in a country that's stuck on the end of an expensive and slow(ish) undersea cable for the past few years, sometimes my thoughts wander towards something like this. Even just a service (US-based) that would download a set of files (HTTP, FTP, bittorrent), burn them to DVD, and post them via airmail would be pretty handy. Bonus points if there was a way to use airline passengers as your couriers, such that the total latency might only be ~24 hours. :)

The airlines have services for that (see Delta Dash) without needing to use airline passengers. :)

Google had a neat feature related to this in implementation but not goal since 2007ish. It was called Google Datasets and let you mail in a hard drive. Google would then host the data on the web. It's audience was purely academic, though... no money exchange involved.

Anyway, it got cut back last Fall. Cool to see it's still living on in some form.

Oh that's a shame, I didn't know it had been cut.

Amazon has a similar scheme "Public Datasets", that store data as an EBS rather than an S3 bucket http://aws.amazon.com/publicdatasets/

New cloud feature: Pony Express

Only kidding this is great for immense datasets.

<lamejoke> Presumably a feature for Django devs</lamejoke>

How do they do virus and security checking? I don't know how they import the data, but it could be that they use a system that can be exploited using a security vulnerability?

It's a (very) long shot for this kind of thing to work, but still it's a risk worth asking about. They are at the end of they day hooking up hardware to their computer - i.e. physical access to the machine.

would you risk your account by intentionally adding malware to your data sets?

HDDs are passive, there is no danger in merely hooking one up. and as long as the file system drivers are solid, shoveling these bits into "buckets" (whatever these are) is safe too. you're just shoveling the bits, not looking at them.

this is about data, not executable code.

They only support SATA and USB-2, which unlike Firewire, do not work by DMA, IIRC. I'd assume they do not support Firewire for exactly this reason: to prevent a device from reading the systems memory directly (known as pwn-by-iPod)

SATA certainly does support DMA. DMA isn't a feature unique to firewire; rather, USB is (nearly) unique in that it's the one high-speed standard that doesn't support DMA.

Also check out Terascale Sneakernet from Jim Gray et al. from 2002: http://research.microsoft.com/apps/pubs/default.aspx?id=6457...

This is going to be great for professional photographers (the industry I work in). I'm been advocating they use S3 for archiving for some time ... now it's easier to get your files there.

hmm... maybe i'll sneakernet all my music/photos now. been waiting for a sneakernet uploading service. it was only a matter of time before they added one. wonder how long before smugmug adds a wraper around this and lets you send your dvd/smartcard/etc media and get it directly uploaded into your account.

Not really "AWS" then!

Dupe: http://news.ycombinator.com/item?id=620089

Although this one is getting the comments.

Probably because of a better title giving people an idea about the service even before they read the article.

The other earlier submission has been deleted and I'm being downmodded. Fair enough, I guess I won't be reporting duplicates any more.

Reply on news.ycombinator.com