Hacker News Reader

Comment by zokier | Hacker News Reader

> It seems no cloud service provider these days is able to offer what was considered an industry standard.

I wonder how many services really had 5 nines availability pre-cloud era either. Somehow I feel your view of it being "industry standard" might be slightly rose tinted

View on HN

Five nines uptime only exists in the mainframe world. Everywhere else it's a requirement set by someone in management, which is met by the vendor in their marketing material. It's never achieved over the long term, but enough time passes so the inevitable downtime can be blamed on the previous management. The vendor meets their "guarantee" by paying back less than a point on the yearly bill, and then everyone can reset the clock and pretend that it won't happen again.

The only people who suffer consequences are the staff forced to work overtime performing SEV0 RED ALERT theater. They will work through nights/weekends while the responsible parties tut-tut and "manage" by reading updates they can collate into the post crisis report. After that, everyone participates in the joy of emergency meetings to discuss said report that will be entirely worthless when a completely different part of the system fails the next time. A more reliable HA solution will be worked up by the engineers, finance will estimate implementation costs, and it will be turned down by an executive on the 8th hole green because they don't care about anything except improving profitability so they can hand themselves a bonus.

Not that I'm bitter or anything.

I worked on multiple services since the late nineties on bare metal over multiple datacentres to achieve 5-nines.

With IaaS that is now easier than ever, yet these so-called cloud service providers dont do any of that - they tie themselves to a single AZ and have ZERO redundancy.

The issue is more like the guarantee is not worth anything.

AWS/Azure/whomever "promise" 5 9s uptime. Something goes wrong, you don't get 5 9s, and what do you get?

A system that went down for 4 hours and a $50 rebate on your next bill!

its not about AWS/Azure etc. They are providing IaaS. Literally compute services littered around the globe. It is up to these so-called cloud service providers, like heroku, to utilize that infrastructure to achieve 99.999%.

I even gave a link in my comment to what AWS say about this.

Are people downvoting me because they dont read, or what?

You make it sound like AWS has 100% uptime and services built on top of them are completely to blame.

And for something like Heroku's managed DBs you can't just achieve 99.99999% availability on a DB without making certain sacrifices. Availability isn't everything past a certain point

has aws ever suffered multi region blackouts? or even all AZs in one?

Thats not what im doing at all. I even gave a link to AWS documentation of achieving 5-nines by utilizing multiple AZs, etc. I also reiterated this in the comment you responded to above..

What sacrifices you talking about when synchronously replicating to a backup environment? Write latency? How do you deal with that usually? How much is too much? There are strategies to deal with reducing replication related latency depending on level of consistency required.

Costs and general complexity. It is quite easy to accidentally reduce a system's uptime by introducing extra complexity involved with higher availability

Oh yeah, there are definitely additional costs and complexity involved.

Im just saying that these cloud service providers offering managed services should be covering all that - they certainly charge as if they do!

And in the case of heroku - and their specific architecture - it is not that complex. Im aware that other cases may vary.

No particular love for Heroku, but you can pay for multi az failover if you want it.

https://devcenter.heroku.com/articles/heroku-postgres-ha

If you don’t pay you won’t get the feature. Given that multi az failover directly impacts their cost that seems pretty fair.

Yeah, I suppose in some circumstances their offering is OK.

I don't think i have ever worked on a system where 10 mins of data loss is anywhere near acceptable though.

I guess for mostly static pages, or self hosting a blog its ok, although id be pissed if i had to rewrite an article. Makes you wonder who their target market is.

Your point about the credit s stands, but could providers don't even offer 5 9 SLAs.

E.G. https://aws.amazon.com/compute/sla/

Less customers, less moving parts, less to go wrong. I'm sure a lot of places were basically rolling the dice, but I'd imagine a lot won that bet while those that lost it had a much more difficult recovery process than today's vendors.

Plenty. But in most cases there was luck involved.

Reply on news.ycombinator.com