Amazon Snowball
aws.amazon.com
aws.amazon.com
When I returned to JPL after working at Google for a year I was tasked with evaluating a Google Search Appliance. We ultimately decided not to keep it, and so we had to erase the disks, which now contained sensitive data. The appliance had a "self-destruct" feature that supposedly erased all the data, but there was no way to verify it. After lengthy negotiations with Google (some people just have a hard time grasping the idea that just because a file has been deleted doesn't mean the data is actually gone) we eventually got them to agree to let us open the enclosure and take out the disks. Forensic analysis revealed that they had not in fact been erased.
Caveat emptor.
"...The data will be 256-bit encrypted on the host [running the Snowball client?] and stored on the appliance in encrypted form. The appliance can be hosted on a private subnet with limited network access."
So I assume the data is encrypted asymetrically.
"...ship it back to us for ingestion. We’ll decrypt the data [using the private key specified in the job,] and copy it to the S3 bucket(s) that you specified when you made your request[/job]. Then we’ll sanitize the appliance in accordance with NIST Special Publication 800-88 (Guidelines for Media Sanitization)."
That links to http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP...
There are a few different types of sanitisation (clear/purge/destroy), and Amazon doesn't specify which type. I assume they would go with "clear", and maybe in a few select places (I'd hope storage media) "purge".
"Clear" is scary though, as for network devices, it is only "full manufacturer’s reset to reset the router or switch back to its factory default settings", and for HDD's it is "Overwrite media by using organizationally approved and validated overwriting technologies/methods/tools. The Clear procedure should consist of at least one pass of writes with a fixed data value, such as all zeros. Multiple passes or more complex values may optionally be used".
So what vector do you want to protect? Accidental data egress shouldn't happen as the data is encrypted. However there are more interesting vectors, such as getting hold of the public key and injecting your own data into another companies buckets...
- Andrew S. Tanenbaum
While you can send in 50TB for $200, taking the same 50TB out costs an additional $1500 charge (50000 * 0.03).
[assuming they are not transferring the data over the Internet, the cost to AWS should be the same or cheaper for reading]
For one thing, the documentation says you can use multiple Snowball devices, but carefully tiptoes around saying whether or not there's an extra charge for doing so. All of the language that actually talks about pricing just says "the device", singular. For another thing, the screenshots of the "create a job" workflow are missing any way to specify that you want multiple devices. It sure looks like one job == one device.
(This reminds me of the pricing issues around AWS's Glacier service. It's not even that the pricing model itself is bad -- it's that the marketing is obfuscatory to the point of being arguably deceptive.)
[0] https://aws.amazon.com/ec2/pricing/
$4300 = 10000 * $0.09 + 40000 * $0.085
According to multiple sources, Internet transit in the US now costs less than 1$ for a 1 Mbps line for large deals, which translates to 1$ for 324GB, which translates to 0.003$ / GB.
Amazon charges 15-30 times that.
(it appears that traffic can be much more expensive in places other than the US and presumably Europe)
You are paying their profit margin, yeah.
It is wildly overpriced, no matter how you look at it. Operating costs even when done on a much smaller and more inefficient scale than for AWS do not make the total cost for incremental bandwidth usage THAT much larger.
Or they start getting older, don't keep the skills sharp enough, and die off. Now you've got to painfully convince them to help you train a new employee to keep the show running, or pay 10 fold your savings hiring the smartest in the world to fix it. But the systems too big, and by the time it's on the "latest and most popular cloud architecture with proprietary systems", you're irrelevant.
But you're right, they make the companies current CEO/CTO look good by cutting costs in the beginning, so who cares right?
I was hoping somebody would bring up Netflix. Netflix is AWS's poster child and they know it. They need AWS still just as much as AWS needs them. The statement you just made proves that. They will do everything up to and including take a loss to keep Netflix around. I can assure you that your company will not be getting the same price quote or technical support that Netflix does unless they can bring them just as many sales by being a status symbol and marketing tool for them.
There seems to be a trend of people thinking "AWS is my friend". No. They are a company, and they exist to make money. Have we not all been bitten enough by this thought pattern and loyalty to learn the lesson of "stay flexible"? I'm not advocating that AWS is done away with entirely and nobody should use them ever. I'm advocating that putting your entire stack into their system is a bad idea and that using "black box" software as little as possible is a better approach. When it was just EC2 it was fine, you can build your stuff on an EC2 box with your favorite flavor of *nix and quickly throw boxes up elsewhere if things go south. Now I'm seeing companies put entire critical infrastructures off on Amazon pre built services like they've never seen a tech company go under, or a fad die, or strongarming with brute power.
This is a complete non-issue for users of AWS unless they are essentially trying to resell AWS services.
> companies that have had their physical goods ideas stolen and are now mass produced at Amazon
AWS is not Amazon, they do different things - see my GP comment replying to you. I also wonder how many times this has actually happened (stealing ideas) in reality, versus the companies simply having an unsustainable business model, or an obvious product. It sounds like the classic case of looking for someone to blame for their own incompetence, and choosing Amazon instead of some other huge corporation or the government, which are also common scapegoats...
If it suited their strategic interest, sure Netflix could get the shaft. Is that scenario likely to play out, seems unlikely. Netflix is one of AWS earliest and most prominent customers.
I would imagine Netflix would leave Amazon long before Amazon shifts their strategy to take out Netflix.
However, AWS is a different thing altogether - it is a set of services that can be used to run parts of your business. Now, if your business involves simply reselling those services, or is predicated the availability and pricing of one of those services being a major part of the value for a service you sell (i.e. the value you add is marginal, with the majority of the product or service's value residing in the service AWS provides) then you are again in a situation where AWS may also decide to offer the same service, and will probably be able to do so more cheaply and profitably.
Again, this would not be a malicious act or directed attack on your business through inside information, and it seems naiive to assume so. You must have been able to determine a market existed and a need could be fulfilled profitably by providing this service, there is no reason a company like AWS could not reach the same conclusion with its vastly larger amount of resources. The possibility of this sort of thing happening should have been determined during due diligence and market analysis anyway, so it should not be a surprise if it happens.
But companies in traditional lines of business using AWS to save money, or using AWS to build a product where the value provided to the customer is inherent in the service provided, not the infrastructure used, are going to be fine. Nobody worries about the electricity company stealing your idea for a product run using electricity...
There are companies offering prices much more similar to cost, for example:
- Hetzner.de servers/colocation offers additional traffic at 1.39 EUR/TB = 0.00158 $/GB
- DigitalOcean offers 1TB traffic with $5/month instances = 0.005 $/GB
It’s done literally nothing.
The high prices seem to persist until one day they don't.
It only takes one of the large players to break the pricing stalemate, and overnight they'll all follow to keep their position in the market.
About 40-50% of that cost was circuits and transit. The vast majority was tied up in labor and equipment costs. If I making money on the whole stack with the market control that AWS has, I would want at least 60% margins on the business -- it's not like locked in customers have easy options.
I think you're going to be absolutely furious when you find out what the component costs of a bottle of any random drink is.
I always found it amusing that hard drives were rated in the hundreds of Gs until someone reminded me that 'time to stop' when dropped on a hard surface was very short indeed...
>It has a Kindle on the side, which functions as an automatic shipping label.
http://techcrunch.com/2015/10/07/amazon-launches-snowball-a-...
Guess what a Kindle's display is?
If you had sticky shipping labels, you'd need some intelligence.
Cloud backup companies have had similar services for a while now, but it's nice to see AWS adopting it.
Someone else in this thread thought $1500 was expensive to get 50TB back out. If you use this for disaster recovery, you could get all of your data back onsite quickly for a very low (comparative) cost, versus trying to provision high speed connectivity.
$10/TB/month? Where else can I store data reliably that cheap? (Yes, Backblaze is half that price. I hope they become a worthy adversary to AWS S3 to drive prices further down).
Yes, S3 is cheap(ish). But given Snowball is a snapshot backup service, it's not comparatively cheaper than it would be to distribute that same data by creating a clone and sending it to a safe place.
S3 is the cheapest "real" business storage option besides Backblaze's new storage offering. S3 can't be compared to shipping disks someplace where they sit offline.
What this offers is a useful way to get TBs of data up to Amazon easily, cheaply, and quickly.
And before the Internet was commonplace, people had to use data lines provisioned by the telephone company to link distant offices, and could also send data that way, as larger companies still do.
There's people out there that will sign a contract under a fake name / address with a phone provider and sell the phones, and the way the providers fight against it usually by running credit checks and verifying address against them. Ultimately, this is very hard to detect when it involves identity theft.
http://www.borrowlenses.com/product/Canon-EOS-C300-Mark-II-E...
Which also makes me wonder why this Snowball device only does Ethernet. Seems like it should have an eSATA port.
EDIT: It appears it supports 10GbE natively. I assume it'll also support lower Ethernet speeds.
Yeah, ethernet has pretty much always downgraded well. That's why a 10GBASE-T interface is perfect :P
(I should have been more clear with my initialisms)
If they use a simple eSATA drive, they have to worry about formatting, layout, etc., whereas an ethernet connected drive with a bespoke client can hide/control all of that for Amazon.
It also gives them easier options for multi-drive, SSD caching, RAID, etc - things they may or may not use today but could without any impact on the end user.
* Also, it's 10Gb ethernet. And how many desktops do you have laying around that will recognize a 50Tb eSATA drive?
"Even with high-speed Internet connections, it can take months to transfer large amounts of data. For example, 100 terabytes of data will take more than 100 days to transfer over a dedicated 100 Mbps connection. That same transfer can be accomplished in less than one day, plus shipping time, using two Snowball appliances."
With a 100 Mbps connection it takes over 100 days [1] but with a 100 times faster connection (10 Gbps) it takes less than a day :)
[1] Assuming no network overhead it is 92.6 days
Obviously this device has been designed to be a multi-time use device.
Amazon definitely do NOT have physical control over this box. They would need to do a complete low level reflash of every single bit of firmware on there, every time it came back. That's not actually that inconceivable in a enterprise grade server, and I hope to see some interesting details about that.
But still... just imagine some of the fun HDD firmware hacks making their way onto this. Or NIC firmware. Or even just the embedded Kindle being rooted, and used to sniff out Wifi networks, report its location via 3G, etc.
Not to mention the obvious data recovery attacks if the disks have not been wiped to the highest levels.
I agree with parent comments, and assume Amazon would put this on its own untrusted VLAN when it comes back. But would they weight it first to see if any pwnies have been inserted into the box? Visually inspect inside to see if physical components have been removed to ensure the weight is actually the same, despite a pwnie inside?
I really hope Amazon put out advice to their customers on how to connect this - ideally it should just be on a point to point link to a sacrificial server containing the data.
Assume Amazon only loads it up with encrypted data over network untrusted, and the customer only takes off the encrypted data over untrusted network.
But yes, if you assume this device is treated as malicious at both ends - just like an unfiltered internet connection, but 10x worse - and that the client software that is doing the load/verification, or unload/verification is doing decent input validation, and your assumption that the user is doing their own encryption prior to transfering it to the device, I agree.
The software load that is wiped every time is a first, and extremely basic, line of defence.
Realistically I'd hope the OS is on a SD card that they can literally take out and throw away after they have the data off (you can pwn the micro-controller on an SD card) - and replace with a freshly baked card.
I used to work for an ISP that would juggle which peering links the usenet servers were favouring to make things more favourable :D
I'm interested in hearing about how they are going to do this.
Does anyone know of a service where I could rent a 20TB device like snowball but not push it to S3?
Edit: Correction, it doesn't appear to be an array, just single device so 1.5-2TB max. Also basically targeted at this SAN solution only.
5 * 6TB drives, RAID5 would put you at nearly 24TB of storage, if my brain is in gear.
He wants to avoid having to have to buy enough drives, etc, to hold all of the data at once
http://docs.aws.amazon.com/AWSImportExport/latest/DG/introdu...
- Kindle's E-Ink
- AWS IAM / KMS/ SNS
- Amazon Carrier? (perhaps in the future?)
- GPS-powered chain-of-custody tracking (AWS working on it, perhaps Amazon Drone delivery in the future?)
I'm curious what the end result looks like in doing this.
http://docs.aws.amazon.com/AWSImportExport/latest/DG/limits....
Edit: And yes this way it's easier for them as well and removes "missing power supplies" (big deal actually by I get the point..)
Are you kidding me? Instead of a 25 cent shipping label they use a $100 e-ink display?
The display alone might get the device stolen.
But in any case, you can just connect it to a sacrificial server on its own network that has nothing else on it.
Out of curiosity, does anyone have a real life example where they send petabytes over the wire? You know, outside the adult industry.
Then you get it shipped to a name and address that are not yours, and you intercept the package before it gets there.
It becomes even harder to trace if the name / address / card all match, for instance if you've taken out a credit card on your elderly neighbor's name
Whether the shippers do or don't keep it, there are much more valuable things shipped every day. Where do you think the gargantuan Catalyst and Nexus switches come from? The line cards in those things probably cost more than a Snowball device.
From my day job, I know this is an issue for mobile operators
I'm still waiting for the big leak on how AWS cooperates with NSA at large.
http://www.urbandictionary.com/define.php?term=snowball
Reminiscent of when Microsoft called an overlay dialog a "floater" and all the South Africans and Brits in the room started laughing.
http://www.urbandictionary.com/define.php?term=floater (the 2nd definition)
Now there is.
No commonly understood dirty version of that, and I suspect most people would've thought it a very cool name as well as appropriate.