Amazon S3 Introduces Reduced Redundancy Storage (2x vs 3x) | Hacker News Reader

Amazon S3 Introduces Reduced Redundancy Storage (2x vs 3x)

docs.amazonwebservices.com

34 pointsjustinsb18 comments

Update: Werner Vogels weighs in: http://www.allthingsdistributed.com/2010/05/amazon_s3_reduced_redundancy_storage.html

Open articleView on HN

Werner Vogels' blog posting on this: http://www.allthingsdistributed.com/2010/05/amazon_s3_reduce...

S4 should respond with a "No Redundancy" feature.

http://www.supersimplestorageservice.com/

A preemptive strike against Google's rumored S3 competitor?

Doubt it. This isn't a big enough feature to be driven by that situation. It sounds like a routine product improvement to me.

I actually think it is. It'll make it harder to say "we're cheaper than aws". Now people will have to qualify that (we're cheaper than some of s3), which makes the statement much weaker.

Well, perhaps they have some corporate spies saying that Google will only store things twice to compete with S3 on cost. Probably not though.

If I use this, how do I know when I need to recreate an object?

From Jeff Barr's blog post: http://aws.typepad.com/aws/2010/05/new-amazon-s3-reduced-red...

"If Amazon S3 detects that an object has been lost any subsequent requests for that object will return the HTTP 405 ("Method Not Allowed") status code. Your application can then handle this error in an appropriate fashion. If the object lives elsewhere you would fetch it, put it back into S3 (using the same key), and then retry the retrieval operation. If the object was designed to be derived from other information, you would do the processing (perhaps it is an image scaling or transcoding task), put the new image back into S3 (again, using the same key), and retry the retrieval operation."

Amazon sells technology that they developed for their own use. I wonder what data they store in RRS? Tracking information? User calculated preferences on products?

map-reduce jobs intermediate results?

I'm probably an aws fanboi by now but I think this is great. Store your not-so-important stuff 30% cheaper. I can see how I'd use this immediately.

Is the price less for this? It doesn't say.

https://aws.amazon.com/s3/#pricing gives the new pricing.

Pricing starts at 10c/GB rather than 15c/GB; hence I figured it's stored in 2 locations rather than S3's default 3 location redundancy.

Does that match up with the 99.999999999% vs 99.99% durability figures? I don't know what the proper math is here.

edit: they do explicitly say "Designed to sustain the concurrent loss of data in two facilities"

I think you're right ... I don't see how these figures are consistent. Maybe they're factoring in the time it takes to restore a copy, maybe 'normal redundancy' S3 is doing something clever, maybe it's simply another instance of Amazon being economical with the truth.

Inexact at best, but ..

  P(3 locations failing) = 1-0.99999999999

  P(1 location failing) = P(3 locations failing)^(1/3) = 0.000215443475

  P(2 locations failing) = P(1 location failing)^2 = 0.999569159

There is no proper math. 99.999999999% is a very silly number. The probability of all of Amazon's datacenters being destroyed in nuclear war within the next year well exceeds 0.000000001%.

Reply on news.ycombinator.com