Hacker News Reader

Comment by nostrademons | Hacker News Reader

Google runs earthquake drills every year in which they assume that the Mountain View & other Bay Area offices have been completely destroyed, all employees within them (including top executives) have been killed, and all West Coast datacenters have been knocked out. Their aim is to ensure continuity of operations. Although each drill usually identifies some deficiencies and potential failure points, in the 5 years I was there the simulation never actually resulted in Google going offline.

The big Silicon Valley companies are all huge multinationals at this point, more powerful (and safer) than many nation-states. I'm not even sure a worldwide nuclear war would take Google offline, although it would likely kill 90%+ of its customer base.

View on HN

> I'm not even sure a worldwide nuclear war would take Google offline

All the major power generation systems in the US and Europe that Google needs to operate, would likely be taken down. They're well known and those would all be intentionally targeted. Google as a shell of its former self might stay up briefly, that wouldn't last long outside of some stray regions. Further, they'd have those stray regions severed by local authorities who would divert the critical power to more important uses (literally keeping people alive). Any resistance by Google at that redirect of their power use would be met by military equivalent force.

Google usually has its own private power generation for major DCs, not connected to the public grid. They're very big on renewables - they've built their own hydro power stations on the Columbia River for their west coast DCs, and they typically use geothermal or tidal power in Europe. Being close to potential renewable energy sources is actually a major consideration for where they put new DCs.

Private property rights might be the first casualties of the apocalypse.

Aren't there hidden datacenters in the wilderness in many countries now? Nothing that a satellite sweep couldn't find, but still.

If you wish to hide, you could always go for abandoned mining works etc. - most satellites won't see that. (Of course, you need to get energy in, so a massive new power line that disappears down a shaft would betray you again ;))

That’s true, but what of solar? Data centers are increasingly being powered using solar panels and other sources.

That is the coolest thing about Google that I have ever read.

You can read more here: https://dl.acm.org/citation.cfm?id=2371516

A new e-mail appears in my inbox explaining that zombies have invaded Georgia and are trying to eat the brains of the data-center technicians there. The zombies have severed the network connections to the data center. No network traffic is going in or out. Lastly, the e-mail points out that this is part of a DiRT exercise and no actual technicians have had their brains eaten, but the network connections really have been disabled.

I’m not sure the servers could handle that little traffic.

You joke, but a sudden drop in traffic like that could very well cause a lot of automated actions to get triggered and a lot of alerts.

Datacenters everywhere would freeze over and the digital ice-age would begin.

> in which they assume that the Mountain View & other Bay Area offices have been completely destroyed

pretty sure both the hayward and the san andreas faults are not capable of this sort of destruction.

sounds more like a dark nihilistic adolescent fantasy.

They're probably not, but the point is to defend against a loss of headquarters and all the resources surrounding it, regardless of the cause. Note that a network partition (where the Bay Area is fine but cut off from communications with the outside world) is functionally equivalent.

There are a number of other scenarios that could conceivably take out all Bay Area offices: nuclear strikes on SF + Moffett Field, wildfires in the Diablo Range that sever all communications, secession of California and following military blockade, large scale protests or civil unrest that do the same, etc. Might as well protect against all of them rather than special-casing earthquakes.

  all employees within them... have been killed

I don't know if I like a response plan that by definition omits any consideration for rescuing people or treating any injured.

As mentioned in a sibling comment, there are a number of scenarios where the people are fine but all lines of communications are down, which is functionally equivalent to everyone being killed:

https://news.ycombinator.com/item?id=16590441

(The actual wording on the e-mail is usually far more colorful, and usually involves a zombie apocalypse. Mountain View employees aren't just dead, they're undead.)

I think the assumption is that if the event actually did involve a danger to life & limb, response would be in the hands of Google Physical Security, local first responders, CalFire, FEMA, etc. Those organizations all do drill for rescuing people and treating injured, and will be in a much better position to coordinate responses than Google ops people.

> more powerful (and safer) than many nation-states

Like maybe safer than Uruguay? Not the US.

I was thinking of Uruguay etc. rather than the U.S. when I made the statement, but given the political climate in the U.S. over the last couple years, I'm not so sure anymore. At this point I think I'd give better odds to Google breaking up the U.S. than the U.S. breaking up Google.

Really. I'd like to see Google take on a war against the entire world. This is something the US is prepared to do constantly. And even win such war (not that there would be much left afterward).

Reply on news.ycombinator.com