Hacker News Reader

Comment by fuzzythinker | Hacker News Reader

Unless Glacier just changed their pricing, how is it cheaper and simpler? Glacier is also $.01/GB storage, but only $.09/GB retrieval, which is cheaper than google's $.12/GB. Glacier also comes with 1st GB retrieval free/mo.

View on HN

Glacier pricing is surprisingly complicated, and the actual cost can be much higher than $0.01 per GB-month if you don't read the fine print.

The biggest gotcha is that you can only access 0.17% of the data you've stored in any given day without extra charges. So if you've stored 1000 GB, you can only access 1.7 GB per day for free.

The cost for going over your daily "retrieval allowance" can be large, because cost is driven by "peak retrieval rate". They find the highest hourly retrieval rate over the entire month, and charge the whole month's retrieval at the peak rate.

This can get expensive fast. Again, with 1000 GB stored, if you retrieve 200 GB over the course of 4 hours, your hourly retrieval rate is 200 GB / 4 hours = 50 GB per hour. Your free retrieval allowance is 1.7 GB per day / 24 hours = 0.07 GB / hour. So your excess retrieval is 49.93 GB per hour.

Amazingly, that 49.93 GB per hour for 4 hours is charged for all 720 hours in the month, so that's 49.93 * 720 * 0.01 = $359.49.

That's an astonishing $1.79 per GB just to retrieve one-fifth of the data from Glacier storage.

So Glacier only makes sense for true cold storage, where you are very unlikely to touch more than 0.00007 of it in any given hour.

Google's Nearline is much simpler and faster for the same storage cost -- and far cheaper if you need to access more than a tiny fraction of the data.

Source: http://aws.amazon.com/glacier/faqs/#How_much_data_can_I_retr...

Thanks for the detailed breakdown, that's really interesting! Sounds like you were bitten by this? :)

Ok. I have to this to this. i'm sorry.

I guess he would say he was... sunglasses FROSTBITTEN YEAH!

I understand, but please go stand in the corner there and think about what you've done.

Don't know about the OP, but we were bitten by this - badly - after our logic to throttle Glacier retrieval failed.

AWS has now added the ability to set a spending limit to avoid runaway retrievals. Nonetheless, from my point of view the ultra-complicated pricing scheme is nothing short of a disaster for Glacier as a product. I think it will continue to seriously impact its uptake, and Google is smart to exploit the opportunity to offer a simple, straightforward pricing scheme.

The use case is people who have a requirement to store data and probably never need to access it, either for reg requirements, or legal requirements, etc. In which case paying a couple of hundred bucks for retrieval is fine, especially if they can access at a high level of granularity

edit: Also as others have pointed out, Nearline is limited in retrieval time as well, so the cost difference isn't nearly as large.

Why does Amazon charge like this? It seems like on the rare occasion that you need to send some person to go grab the tape/disk from storage and bring it online, Amazon would want you to get all the data you need and put it back in storage.

Incentivizing users to bring the data online once a day to trickle it out seems bad for all involved.

I think it makes most sense to think of it as paying for access to the tape robot.

If Amazon store your data in a tape archive (I don't know if they do, but they at least seem to have similar constraints), they can only access a small portion of the stored data at a time, so they need to control how often people request data.

They could just rate limit everyone, but this way allows people to pay for priority in an emergency while still discouraging everyday read requests.

The pricing makes more sense if you're a large user with data spanning several tapes than if you're in the single terabyte range, but the low limit still discourages you from making requests causally, which helps them keep their SLA.

If they predict that you'll trickle out your small file, they can just read out everything on the first access and cache it online, so there's no extra trips to the archive for them.

You assume Glacier uses tape storage... But the same principle still applies, they're trying to prevent people clogging up the network so when somebody does need to do an emergency restore there is lots of spare network capacity.

Note: The following is speculation best I know. I'm recalling from memory what I've read on the internet written by someone who did not have a direct source.

Glacier uses low-speed (5400 RPM) consumer drives, which they then clock down lower to save energy. Any given 'rack' only has enough power to power a few drives on that rack, the rest is powered down.

To prevent multiple customers from all trying to pull their data out they needed to introduce a rate limiting system, which they did with this exorbitant pricing.

Except for the Glacier retrieval fees which can be insane if you choose to retrieve more than 0.17% of your stored data per day.

https://aws.amazon.com/glacier/faqs/#How_much_data_can_I_ret...

But if you retrieve the same 1000 GB from Glacier in the same 3 days that Google takes you would pay a 98 dollar retrieval fee. And Amazon would charge 300 dollar less traffic cost ($ 0.09 vs $ 0.12 / GB for 1000 GB) so in the end Glacier is still 202 dollar cheaper AND allows you to retrieve the data much faster if you really have to at a certain price.

So Google looks cheaper/simpler, but the amazon model is actually quite good for disaster recovery where you may be in a situation where you "need it now at any cost"

Internet egress for 1000 GB is $120 for Google and $90 for Amazon, a difference of $30 not $300.

Glacier have additional fees depending on how fast you retrieve the data. Honestly I'm not sure what they are because I've never had to use it, but my impression is that these fees could be quite high.

At least with Glacier I get the option of retrieving faster. Nearline is going to take more than 3 DAYS to retrieve my data if I only have 1TB stored (3MB/s).

Let's assume that with Glacier you can retrieve your 1000 GB in 4 hours instead of 3 days.

You will be surprised by a retrieval fee of 1000 / 4 * 720 * 0.01 = $1,800.

If you want to get your 1000 GB for free, you'd have to request no more than 69 MB per hour. It will take you about 20 months to get your data!

With Glacier, limit retrievals to less than 5% per month / 30 days / 24 hours = 0.007% of total data stored per hour to avoid these ugly surprises.

Source: http://aws.amazon.com/glacier/faqs/#How_much_data_can_I_retr...

For many people waiting 3 days is more attractive than paying the ~$1000 that it would cost to retrieve it in one day from Glacier.

For disaster recovery 3 days might mean the end of your business.

For disaster recovery $1000 is peanuts, that's a company thing, GP mentioned 'people', not companies, different use-case.

I wrote a backup program that uses Glacier, and the retrieval policies are nearly impossible to manage and explain. But the thing I really don't get is that you can use Amazon Import to read a hard drive into Glacier, but you can't use Amazon Export to get a drive full of data back out. You can with S3.

In a disaster situation, a company is going to want hard drives sent to them next day. As others have mentioned, this isn't a money thing, it's a time issue. But it isn't available with Glacier (probably not with Google either...)

If you're willing to pay more for faster retrieval I guess you could store e.g. 5TB of random data to get a fast "base speed"- cutting those 3 days down to 12 hours.

That would be more expensive than just storing 1TB with the normal, full speed Cloud Storage service.

Or use Nearline for archival data as well as disaster recovery. You won't need to transfer the archival data for normal disaster recovery scenarios, so it'll be -- for that scenario -- extra data that boosts your speed, but it's still, on its own, a useful use of the service.

They can be. It's possible to set a retrieval policy on your bucket either at free tier, or a fixed price point so you can control the cost of your final bill if you want to retrieve faster than the free tier

"Unless Glacier just changed their pricing, how is it cheaper and simpler? Glacier is also $.01/GB storage, but only $.09/GB retrieval, which is cheaper than google's $.12/GB. Glacier also comes with 1st GB retrieval free/mo."

As long as we are comparing effective pricing, which is the only number that matters with Glacier and "Google Nearline" (and, to some degree, S3) it should be noted that rsync.net PB-scale is 3.0 cents with no additional charges.

That is, the effective price, no matter what your use-case or traffic/usage, is 3.0 cents per GB.[1]

Of course, you do have to buy a petabyte of it ...

[1] http://www.rsync.net/products/petabyte.html

Not to nitpick, but your CEO page is a bit misleading.

    > "a two year contract is required." [1]
    > "There are no contracts, overages, fees, or license charges at rsync.net." [2]

[1] http://www.rsync.net/products/petabyte.html

[2] http://rsync.net/products/ceopage.html

Reply on news.ycombinator.com