Hacker News Reader

Comment by jedgrant | Hacker News Reader

Regretting the decision to go with Azure. Talk about terrible timing. We have media outlets interested in our site, we send info and the site is dead. Talk about a crap first impression.

View on HN

It is totally frustrating, but at the end of the day similar outages happen with all cloud providers.

Looks like Azure has been down more hours in the past week than AWS has been down all year.

https://cloudharmony.com/status-1week-for-azure

https://cloudharmony.com/status-1year-for-aws

Can you point to an example of this happening on AWS in multiple regions simultaneously?

Yes. The day half the internet seemed to be down. It took Amazon more than 24 hours to recover, and having your services in multiple availability zones did not shield you from the failure.

http://aws.amazon.com/message/65648/

This seems to have only affected one region. Am I missing something?

Yes. It started as a failure in one region, and propagated to others as it overloaded the "control plane" -- the stuff that runs "the cloud", and EBS tried to replicate "failed" disks to the point that Amazon ran out of disk space in the cluster. At the time, I was paying for RDS Multi-AZ which runs your database in multiple availability zones at once with hot failover if the primary goes offline. It failed to fail over despite that. Many large sites went down for a very long time that day, and people couldn't spawn replacement instances even in other AZs than the one the failure started in.

You're confusing region with AZ. They've never had a multi-region outage (yet).

It was one region, multiple availability zones. You're right Multi-AZ != Multi-region (for things like Sandy and natural disasters) BUT they have mostly separate infra which does make multi-az failures very unlikely. Due to its simplicity (with VPC and stuff) some people (perhaps wrongly) treat multi-az like multi-region.

Ah, cool, thanks for the info.

Interesting to think about the potentially compounding failure modes these services are dealing with.

I'll have to look that incident up to check out their postmortem vs. what Microsoft ends up putting out.

Thanks again ~

I feel your pain. My startup was featured on TechNet today, I was quoted saying how great Azure worked for us so far... lots of folks were checking us out, and then Azure went down hard, taking our production systems with them. Talk about negative publicity.

Reply on news.ycombinator.com