Azure services in South Central region down | Hacker News Reader

Azure services in South Central region down

azure.microsoft.com

23 pointschadash11 comments

I'm quite surprised of how little noise this issue is generating. Not sure does that mean that the issues are actually very localized (although I'm not sure they are, it is not entirely correct that only one region is offline), or that Azure is not used so much so people don't care.

I was really surprised too. It must be the intersection of azure devs, working in South Central, and who are on hacker news is pretty small.

Azure is the #2 cloud platform on Earth, so your last statement is bunk. The outage is very localized, none of my instances in South Central were ever "down," there was a brief 5-10 time frame when I noticed CPU spikes, I assume from a rebooting process from mitigation...which effected some availability but it was never truly down.

The problem was caused by a power surge to their cooling system, which eventually led to systems shutting down to prevent overheating.

The main thing is the spill over from authentication, which is effecting other regions. This isn't an AWS/GCS type even where every service suddenly cascaded. My portal is still not 100% functioning, but all of my services are operating normally and are up. So it's probably at this point only effecting AD and provisioning rather than service SLA.

Loading the Azure status page doesn't work in Florida. Apparently you can VPN (assuming you do not choose a location that maps to South Central) into Azure and migrate your infrastructure if needed.

Unfortunately, also AzureAD and Azure Resource Manager show impacts.. This is pretty substantial impact. South Central Us might have been the specific location, but the impact is causing headache for just about everyone.

South Central has been down since early morning; around 4/5 AM Central. I am really surprised and disappointed that they haven't been able to get it back yet.

This is not isolated only to South Central. We have issues accessing to resources in all US/European data centers, we can't authenticate.

I can confirm. We are having authentication issues as well.

Status page is intermittent and when loading may not be correct. It’s showing the VSTS issues but not VM issues in US West

According to Microsoft Service Page (in case people can't access it)

Starting at 09:29 UTC on 04 Sep 2018, customers in South Central US may experience difficulties connecting to resources hosted in this region. Engineers have isolated an issue with cooling in one part of the data center, which caused a localized spike in temperature, as the preliminary root-cause, which has now been mitigated. Automated data center procedures to ensure data and hardware integrity went into effect when temperatures hit a specified threshold and critical hardware entered a structured power down process. Engineers are now in the process of restoring power to affected devices as part of the ongoing mitigation process.

Some services may also be experiencing intermittent authentication issues due to downstream Azure Active Directory impact, and engineers are separately working on mitigation options for this also.

The next update will be provided at 15:00 UTC or as events warrant.

Reply on news.ycombinator.com