Hacker News Reader

Comment by monochr | Hacker News Reader

And this shows us why it is a bad idea to have a large portion of the worlds digital infrastructure in a place which will destroy it sometime in the next century.

Put it in the middle of the Australian outback, nothing's happened there for a billion years, nearly literally.

To the people down voting me: please explain why you think it's a good idea to put our digital infrastructure in a place which might well be destroyed in an afternoon when it can be put quite literally anywhere.

View on HN

Yes, this point was also raised by Maciej Ceglowski (hn: idlewords) in the talk that keeps circulating in HN comments:

http://idlewords.com/bt14.htm#america

"And speaking of California, let me try another tack.

This festive map shows seismic hazard in Northern California, where pretty much all the large Internet companies are based, along with a zillion startups. The ones that aren't here have their headquarters in an even deadlier zone up in Cascadia.

Now of course, each company has four zillion datacenters, backed up across the world. But how much will that matter when there's a major quake, and Silicon Valley can't get to work for a month? All of these headquarters are going to be shut down for a long time when the Big One comes. You're going to notice it.

So even if you don't agree with my politics, maybe you'll agree with my geology. Let's not build a vast, distributed global network only to put everything in one place!"

> But how much will that matter when there's a major quake, and Silicon Valley can't get to work for a month?

Well, putting everything in the Australian outback doesn't help that much, because there are good reasons why thousands of tech workers can't work there. This is essentially an argument against having any population centers in earthquake zones.

> This is essentially an argument against having any population centers in earthquake zones.

Which is an interesting point in and of itself.

Is it? What American city is not in an earthquake zone, or a tornado zone, or a hurricane zone, or a blizzard zone?

Phoenix. Sure they have drought, but not much other weather or natural catastrophes to speak of.

Okay, that's one. Where do we put the other 300 million people?

http://www.reddit.com/r/dataisbeautiful/comments/29gw4s/worl...

The problem with Phoenix is that it already has a larger population than its water supply can support.

Atlanta is right on the edge of all of these, but it's rare that they come this far with enough force to matter.

Indeed. In fact, it seems like the Pacific Northwest is a pretty solid choice, as is the Bay Area, although maybe the NY Times isn't properly weighting the relative risk of very bad earthquakes:

http://www.nytimes.com/interactive/2011/05/01/weekinreview/0...

Supposedly that is why AWS located their infrastructure in Oregon, after starting in NoVa. They optimize for "lowest probability of disaster" before considering other options like cost (primarily power) and connectivity.

While I don't remember the relative probabilities, the next big earthquake in the Pacific Northwest and Northern California region is expected to be much more intense: https://en.wikipedia.org/wiki/Megathrust_earthquake#The_Big_...

In the list of worst predictable natural disasters in the Continental US, it rates one above the expected earthquakes to the south, the result of two plates mostly sliding past each other: https://en.wikipedia.org/wiki/San_Andreas_fault

Worse, in part because we aren't at all prepared for it (aside from I hope more serious buildings) would be a return of major earthquakes in mid-West: https://en.wikipedia.org/wiki/New_Madrid_Seismic_Zone (something I pay attention to since that's on the other side of Missouri from me).

I forget one of them, and the worst, which few in the US would survive, and where being in Australia might not be a bad idea at all, would be a return of the Yellowstone Supervolcano: https://en.wikipedia.org/wiki/Yellowstone_Caldera

Pretty confident if there were a 1989-level earthquake, most startups and tech companies would just decamp to somewhere else for the recovery period. The easiest is when you already have an office somewhere else (we have London and soon Asia, although SF is still far larger); I'd probably pick Hawaii or Seattle personally.

It's easy enough to find an off-season vacation destination and move everyone there; essentially anywhere is cheaper than SF, so that's easy. Not every company has to move to the same place.

It'd suck to be a service-industry worker in SFBA after an earthquake, but tech employers can just move, and hire service-industry people to fix up their offices for a few months before returning (or not).

(this might not be necessary after a 7, but after something insane like a 9, would definitely happen)

All of the companies that don't already have away-from-the-Bay data centers will be begging for hardware. And if your name isn't Microsoft or Oracle or some other big ams, you'll be on the end of a product delivery pipeline measured in months. Hardware is going to be fought over.

If all your data is in your own building, you might be okay (assuming you can get at it, stuff it into a truck and move it out). Otherwise you're going to be waiting for a while.

No one really has servers In the Bay Area anymore, in my experience. A few startups with legitimately special security needs and competence (Stripe), but mostly people either use the cloud (not in the bay) or colo/managed hosting (also not in the bay, usually).

There might be a run on laptops, monitors, chairs, etc which overwhelm the Hawaii and Las Vegas retailers for a while, but basically California's high energy costs, low power circuit availability, high sales/use tax on servers, etc have solved this problem already.

There are plenty of servers in the Bay Area. There are several large colos in the South Bay (Equinix has several facilities in San Jose), San Francisco has a large colo at 200 Paul Ave, there's a bunch of stuff in the East Bay as well. Amazon has a nothern california region, not sure exactly where those servers are, etc.

Sure, I have equipment in several of those, but it isn't a sole hosting location for most large tech companies. Really big companies tend to be multi site. Smaller ones use cheaper clouds or managed hosting generally outside the area. A lot of the bay area gear is enterprise for local companies which would already be screwed by an earthquake, or network, etc to support people who live in the Bay Area.

SFBA is critical for personnel, not so much manufacturing or hosting.

There is about 3mm square feet of datacenter space here. There are very few large companies which have servers only here and not somewhere outside the area but which would be reasonably expected to continue operating if their servers were somehow unaffected.

Okay, but you started by saying "no one really has datacenters in the Bay Area anymore", and now you're talking about sole hosting, which is a completely different assertion.

True. (You may also note the time of my initial post; I was still awake at 0320, and this was shortly after the quake...); I wasn't being particularly precise.

However, SFBA isn't even the first location for most companies I see. They go either into the cheaper AWS cloud regions (us east or the Oregon), or managed hosting somewhere (rarely SFBA).

Bay Area companies which directly get colo early on are fairly rare: stripe, square, etc. I fully support it as a strategy, but it is statistically insignificant.

Aside from price, east coast single location also gives you a lot better latency to Europe. Asia is usually screwed anyway, but the extra 80ms makes a big difference.

In 1989, most of SF was back up and working by the next Monday. Of course, that was a different era without much telecommuting, and many people suffered tragic losses, but it wasn't bad enough to "decamp" anywhere. The oldtimers seem to complain more about the world series than the quake.

1989 was also only a 7.1. Go up almost an order of magnitude to a 7.9 or 8.0 and it might look a bit different.

That wasn't a particularly large earthquake.

Do you run periodic disaster recovery tests including simulating one of your location being offline (including employees being unreachable) for e.g. one day? This is a good test to check whether the know how is well balanced geographically.

I've noticed a lack of of human planning in most DR plans. In some industries it's called business continuance. Our company has focused mainly on the technical side of the business: backups, DR, offsite access.

Depending on the situation, earthquakes can bring fires and tsunamis. So what do you tell the network engineer that had their house burn to the ground? "Sorry for your lose; we're decamping for Las Vegas. Wheels up in 8 hours"?

If the big one does hit there is a lot more at stake than just your precious data.

I imagine for a large tech company, it would go something like this:

1. Immediately call extra hands in other locations (datacenters, customer service, etc); begin over-time scheduling to alleviate need for workers in disaster zone. Doesn't require any coordination of disaster zone, just remaining executives located outside.

2. Offer two special payments to workers in the impacted zone: 1. Disaster recovery assistance, to help them cushion emergency needs and get their family/belongings/etc recovered faster; 2. Relocation bonus, to create an incentive for quick relocation to temporary (or new) office.

3. Spend as much as you to have to get extra emergency services in to the area, and generally assist in government/regional recovery area. This will free up the last of your staff for temporary relocation.

I doubt that a large company (we're talking Google, Microsoft, Facebook, etc) would take more than a couple days to have other sites working overtime to cover the load, less than a week to have already transported some of their staff to a new site, and less than a month to have brought almost all of the office back online.

This timeline, of course, assumes that the problem wasn't that their office fell on all their employees, killing them or something of that nature.

Likely, we'd see a hiccup in the development cycle, but in terms of keeping infrastructure online, emergency bug fixes, etc, we'd probably see relatively little damage.

Disasters tend to have a knock on effect that can ripple for years. 2 that come to mind were the floods in Thailand and the 9-11 terrorist act. The floods temporarily knocked out the supply chain of HDD components, which had the knock on of system builders not having enough components. And eventually, technicians not having enough spare replacements for current systems.

1. In most businesses these other locations are supplementary to HQ. Typically there is no one at the datacenter and CS centers rely on guidance from HQ. They are not in the position to make decisions and drive the company.

2. Growing up and working in Florida you natural disasters are like clockwork. I can say for sure that 1 is covered by insurance and the gov't and 2 is going to be hard to pull off without looking like a complete dick. Also, you have other constraints. Airports may be closed and driving can be dangerous. Plus cars need fuel which is typically in short supply and expensive.

3. If you have a company of a few thousand I would say a few hundred are essential. Where are you going to find a place to house a few hundred people at once? Remember that other companies are thinking the same as well.

I also believe the most essential employees are that way for a reason. If they have the same preparedness in the company they also have the same mentality in their community. It would not be illogical to imagine the person in charge of DR for the company is also a member of the volunteer fire department.

Microsoft would be able to recover, then they find out the caterers don't have food to serve the cafeteria, employees can't come to work because there is no fuel, etc. These are the soft issues that most DR plans don't account for.

Google has a program called DiRT (Disaster Recovery Testing)[0], when in a certain week of the year, some major exercises that simulating the loss of headquarters are carried out, including the (simulated) loss of some teams, some network connections in the headquarter and/or the physical access to the headquarters buildings. That makes a lot of sense when you place your headquarters at a place that can be demolished by an earthquake.

[0]: http://queue.acm.org/detail.cfm?id=2371516

That is actually an excellent idea. Loss of manpower is something that is overlooked in a lot of companies. I believe part of it is job security (make yourself invaluable). And part of it is HR optimisation. Why pay for 2 people to do the same job when 1 will do? The same reason you pay for 2 servers, 2 backup processes, etc.

Not a case of "if" but when.

Used to work in Florida, and it's a pretty noticeable difference in DR/Continuity strategy to what I work with now out here (SF). For the first 24 hours after Hurricane Ivan it was completely focused on: Are our employees safe; do they have water/food; what medical aid is needed; how can we get blankets and cots into our office so our employees have a place to sleep tonight? All the roads are washed out, there was no real in/out of town for a week, no power to our main DC for 3. Having redundancy in Chicago and Virginia didn't really mean shit when we couldn't get key people to those data centers to run them full steam.

The end result was a high reliance on contract employees at the fail-overs, and a real appreciation for the human element.

Yea. In real life it doesn't quite work that way. You would be pulling families apart ("Honey, you stay at home in Palo Alto, fix the house, make sure everything is okay with the kids while I go to Hawaii for 2 months") isn't gonna fly.

But then again, I keep forgetting that the 'real' valley is just for early 20s-fresh-out-of-school-startup-single-guy-whizz-kids apparently.

It need not be a 1989 level earthquake. That's the biggest worry here.

It wouldn't be Hawai'i (as much as I'd love that). It'd almost certainly be LA or Vegas. Maybe Phoenix.

... seismic hazard in Northern California, where pretty much all the large Internet companies are based, along with a zillion startups. [..] All of these headquarters are going to be shut down for a long time when the Big One comes. You're going to notice it.

What we'll notice is not having filters for the pictures of our Starbucks cups. The demographic of teenage girls in malls will be devastated.

Silicon Valley is no longer the epicenter of technology; it's the epicenter of the fashion of technology.

Your comment is both incorrect and childish and this is actually an important and interesting topic, please don't participate on HN discussions with comments like this.

Comparing the startup scene in silicon valley to fashion is actually very apt. The loss of life and pain would be devastating, but lets not think that a few billion dollars worth of hardware would be anything other than a tax write-off and some middle managers pain to deal with for the next year.

If you want to look at devastating datacenter loss, if anything happened to Northern NJ the markets would go into a tailspin.

your comment is no better than op's. you could provide some examples why his opinion is wrong, because i believe quite a lot of people (myself included) think that most of companies that only exist in silicon valley are about extracting money from markets that are essentially driven by fashion.

You are right, I should have made a better comment or no comment at all.

I'm just tired of the obviously incorrect proposition that Bay Area is producing just frivolous photo sharing and social communication tools (which both has a huge value for humanity, but that's a different story) and I seriously find it an interesting thought experiment what would happen on a global scale if the Bay Area would be devastated by a big earthquake.

The obvious tech giants like Google, Apple, Oracle and Cisco are arguably supporting a huge number of businesses and organizations around the world. But a glance through a list of just publicly traded US SaaS companies reveals that about 40% are from the Bay Area. These offer SaaS-based solutions ranging from HR tools (Workday), IT service monitoring (Splunk), ITMS (ServiceNow), CRMs (SalesForce), ERPs (NetSuite), Insurance backends (Guidewire) to supply chain management (e2open), to just name a few.

If Bay Area suffered anything resembling 2008 Sichuan earthquake, it would halt a good part of the economy for a while.

Could you explain the phrase "fashion of technology" more?

It sounds like you're making an over-generalization based on the last few years of "social" while overlooking many of the largest tech companies in the world. Apple, Oracle, HP, Cisco, Google, Intel, the list goes on, are all headquartered here. http://www.siliconvalley.com/SV150/ci_25548370/

What made SV great was that it was, up to the 80's, the place where the foundations of technology were created. We've been building on those foundations since then and we've come to the point now where it's clear how frivolous it's become. We can't take it any further so we're adding pastel colors and glitter. It's become a series of fads. The 'great' tech companies of today's SV are no different than the 'great' lines coming out of Milan this season. Both will be gone when winter arrives having added nothing.

It was a poor place to make the comment; it wasn't on topic. My apologies. But when the comment was made that zillions of startups would be wiped out, and that we would notice, I wanted to clarify that we might notice but only in the most superficial way possible. Yes, it's a generalization -- there are always exceptions, and there are large tech companies there, but there are large companies in every major metropolitan area. If HP can't brand laptops for a year, I think we'll somehow manage to all survive.

I made the comment because I wanted to correct what I saw as an antiquated notion: that SV was different from Houston or Singapore. It used to be. It's not anymore.

> You're going to notice it.

If the whole of the Americas was wiped off the Internet tomorrow then, for those of us in The Rest of The World (TM), life would go on. I think the only web property that would be annoying to not have would be Wikipedia. Second to that is the Google search engine but I take it as a given that they could cope just fine if America just mysteriously vanished one day.

I think that there are some that think that Silicon Valley is the centre of the universe, some indispensable magic place where all of the innovation happens. If there was The Big One (for the valley) we would soon find out exactly how important things like social networks and iphone apps really are. Life would go on. If anything, a big earthquake in Caliphonea is far preferable to a big eathquake in Japan. In Japan they make stuff the world needs.

We would certainly not cope "just fine". Sure we'd get over it, but there's plenty of stuff depending on American services. The whole .com/.net/.org registry is just a small example.

  If the whole of the Americas was wiped off the Internet 
  tomorrow then, for those of us in The Rest of The World 
  (TM), life would go on.

????

What about the DNS roots, all the sites running on AWS, any site using Cloudflare or the JQuery/code.google.com CDN, etc

can't tell if sarcasm or not, so here you go.

>DNS roots

http://www.iana.org/domains/root/servers

>AWS

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-reg...

>Cloudflare

https://www.cloudflare.com/network-map

>code.google.com CDN

https://developers.google.com/speed/libraries/devguide#Intro... - no location list though, only "globally available"

>To the people down voting me: please explain why you think it's a good idea to put our digital infrastructure in a place which might well be destroyed in an afternoon when it can be put quite literally anywhere.

For one, insensitivity. There could be some victims even today, and you're talking casually about an event that will wipe out the Valley and S.F.

Second, most data centers are in tons of places for redundancy. Google, Microsoft, Amazon, etc etc. Even low cost providers like Linode And Digital Cloud have redundant data centers in 3-4 locations around the wolds.

Third, ever done a ping to Australia? When most of your users are in the US, it's not really a good idea to have data travel halfway around the world and back. Even if anything else improves, the speed of light will remain constant.

Fourth, you think Valley based datacenters are not built to endure earthquakes? That they are just some dumb houses or something with racks inside?

>For one, insensitivity. There could be some victims even today, and you're talking casually about an event that will wipe out the Valley and S.F.

I don't see why tragedy should stop us from thinking rationally about the situation.

>Second, most data centers are in tons of places for redundancy. Google, Microsoft, Amazon, etc etc. Even low cost providers like Linode And Digital Cloud have redundant data centers in 3-4 locations around the wolds.

The problem is that something between 1/3 to 1/2 of all connections to Asia pass through California[1].

>Third, ever done a ping to Australia? When most of your users are in the US, it's not really a good idea to have data travel halfway around the world and back. Even if anything else improves, the speed of light will remain constant.

The machine I have in Sydney pings 60ms to MIT on average. Caltech, UWS and UCLA are all down from there which probably has something to do with the Earthquake.

>Fourth, you think Valley based datacenters are not built to endure earthquakes? That they are just some dumb houses or something with racks inside?

Are the cables made of magic unicorn hair that doesn't break? Because if [2] or [3] happens I don't care how much like a bunker your data center is, it's not talking to anyone. Which also leads back to the problems in number 2, namely, far too much bandwidth passes through California for how dangerous it is.

[1] https://www.technocrates.org/wp-content/uploads/2012/10/subm...

[2] http://www.geerassociation.org/GEER_Post%20EQ%20Reports/Ital...

[3] http://www.cloacina.org/blog/wp-content/uploads/2012/03/crac...

I'm curious how your pings are basically as fast as the speed of light. Do you have a direct fiber cable between sydney and MIT with no repeaters or intermediate switching equipment, and photons that travel at twice the speed of light? Because nobody does.

Ping measures round trip time, not point to point.

Distance from Sydney NSW to Boston, MA: 16,230 km

http://en.wikipedia.org/wiki/Optical_fiber: "From this information, a simple rule of thumb is that a signal using optical fiber for communication will travel at around 200,000 kilometers per second. "

Some basic math: (16 230 km) / (200 000 (kilometers per second)) = 81.15 milliseconds

that's one way. You should be seeing no bnetter than 160ms, and even that is idealized because there is switching equipment that induces delays.

$ sudo ping -f www.mit.edu

PING e9566.b.akamaiedge.net (184.24.249.86) 56(84) bytes of data.

....^C

--- e9566.b.akamaiedge.net ping statistics ---

728 packets transmitted, 724 received, 0% packet loss, time 10177ms

rtt min/avg/max/mdev = 25.935/38.167/131.406/16.421 ms, pipe 11, ipg/ewma 13.999/35.015 ms

http://www.geoiptool.com/en/?IP=184.24.249.86

My the server I'm testing this on is in Sydney, the server info on the geoip says I'm pinging somewhere in Cambridge, Massachusetts.

There are many explanations which are simpler, that don't require faster than light photons.

In particular, your network path is sending you to a server which is in australia.

$ host 184.24.249.86 86.249.24.184.in-addr.arpa domain name pointer a184-24-249-86.deploy.static.akamaitechnologies.com.

trimmed mtr output:

19. sun1-ge5-0.gw.optusnet.com.au 57.1% 8 186.4 184.9 182.1 186.4 2.3 20. 198.142.144.70 42.9% 8 186.9 182.6 180.3 186.9 2.9 21. sun1-ge12-0-0-500.gw.optusnet.com.au 0.0% 7 179.7 180.3 179.5 182.3 0.8 a184-24-249-86.deploy.static.akamaitechnologies.com 22. 198.142.144.70 0.0% 7 185.6 184.6 179.5 194.2 4.9 a184-24-249-86.deploy.static.akamaitechnologies.com 23. a184-24-249-86.deploy.static.akamaitechnologies.com 0.0% 7 185.1 184.0 178.8 187.8 2.8

that's a cache of MIT's web site hosted in Australia (Akamai edge cache)

http://research.microsoft.com/en-us/um/people/ratul/akamai.h...

geoip is worthless in many situations, this being one of them.

Also, most routers implement ICMP on a slow path (not on the data plane) so the reported latency can be higher than the path latency.

The real wtf is that you need root to ping.

Sending ICMP traffic requires a raw socket which needs root to open. To that end, the ping command is normally installed suid root, though these days there are ways to selectively give programs extended privileges like that. Sometimes people do system hardening which removes the setuid bit on ping, requiring sudo to make it work.

What do you know... TIL.

Check your local linux box with:

  file /bin/ping

and you will see that it is setuid to root.

I think it depends on the distribution: https://wiki.archlinux.org/index.php/Capabilities seems to imply that ping could be used with CAP_NET_RAW capability instead of setuid root. http://blog.siphos.be/2013/05/capabilities-a-short-intro/ suggests that as well.

The -f (flood) option requires that you be root, even when the binary is setuid root.

>I don't see why tragedy should stop us from thinking rationally about the situation.

I don't see why thinking rationally would entail talking without any sensitivity and humanity -- which is what I complained about, not the "thinking rationally" part itself.

>The machine I have in Sydney pings 60ms to MIT on average.

Doesn't 60ms sound quite bad as the optimal average one can have?

>Are the cables made of magic unicorn hair that doesn't break?

No, just that buildings are make of scientific structural construction that endures and absorbs earthquake pressure.

As for the cables, that's an easier problem to solve, and it will be more localized.

The main digital infrastructure that matters is people. The USGS estimates [1]:

"A repeat of the 1906 magnitude 7.9 earthquake, the worst case scenario for the Bay Area, is estimated to result in about 5800 fatalities if it strikes during working hours. Most scenarios, however, have maximum projected fatalities on the order of several hundred, reflecting the success of earthquake-resistant design and construction practices in California, particularly in residences. The loss of life is predicted to be highest if an earthquake occurs in the early afternoon when people are working in commercial buildings with varying vulnerability to quakes These predicted mid-afternoon fatalities are generally about 5 times higher than values predicted at 2:00 AM when the population is assumed to be in wood frame residential units."

[1] http://earthquake.usgs.gov/regional/nca/wg02/losses.php

Latency and Bandwidth. You want your Data Centers to be close to improve performance. You also want to have massive amounts of bandwidth - Australia is particularly bad for both.

Also - there are a ton of data centers out in Nevada, Oregon, DC, etc...

Nielsen's law means that we have to build half the pipes we have every year. It doesn't have to be Australia, just some place where natural disasters don't happen. The places you mention don't warrant much more optimism than California:

http://www.usa.com/nevada-state-natural-disasters-extremes.h...

http://www.usa.com/washington-state-natural-disasters-extrem...

http://www.usa.com/washington-dc-natural-disasters-extremes....

Could you explain Nielsens law ? Build half the pipes to keep growing - is this like Moores law? Nielsen just triggers UI design in my synapses you see

Pretty much[1], but the growth is at 50% a year as opposed to 60% so networks get slower compared to processors from the same generation, combine this with Kryder's Law[2] which states that magnetic storage grows at something like 100% a year and we have the fact that the internet is most likely a fad which will pass once we can store essentially all the information we could possibly need in a convenient physical package.

It will be literally cheaper and faster to send homing pigeons with hard drives around their necks than to have to wait for the same amount of data to be downloaded [3] and [4] will become a viable way of implementing ip.

Probably the first industry to be hit by this new disruption will be music or books when every song/book in the world could be stored on a phone. For example the spotify library at 320kbps would take up something like 250 Terabytes, which will be the size of a laptop hard drive in 8 years give or take.

The upside to this is that movie studios will probably push 16k screens and extremely high frame rates to keep files big enough that you can't just have one drive with a few thousand movies on it.

[1] http://www.nngroup.com/articles/law-of-bandwidth/

[2] https://en.wikipedia.org/wiki/Mark_Kryder

[3] http://www.smh.com.au/technology/technology-news/carrier-pig...

[4] https://en.wikipedia.org/wiki/IP_over_Avian_Carriers

The Internet is not a fad.

You can easily store all of Hacker News locally, at our computer, so why do you come here? For news, obviously. You can't have news pre-saved at your computer by definition, and you simply won't accept (nobody does) the few-hours ping time of the sneakernet.

By the way, carrying media around was always faster and cheaper than delivering the data through the Internet. The trend we are getting is exactly the opposite of what you described; more people are choosing the net option, because although it gets relatively more expensive all the time, in absolute terms it's getting good enough.

You're going to have to deal with the latency anyway. I'm not suggesting Australia (might be great for archiving stuff), but getting 20ms away from the Bay would be good for you, and if you're algorithmically tied to a close geography, better for your product.

Australia isn't an ideal location for keeping data center cool either.

If you splash out for enough PV, you can quite likely generate enough electricity to power a DC, cool it, and have some left over. That's before trying fancier stuff like solar chimneys and the like.

The infrastructure of many companies is not located here or may be distributed geographically. Its in each company's self interest to plan for disaster. And if they don't, their competition stands ready to fill the gap.

Of course, things like the power grid and water systems where there are no competing providers, those are more likely to leave people completely without service.

Even the incentives to be robust or even safe are greatly reduced. A company with a legally protected monopoly can even blow up half the houses on a street (as PG&E did in San Bruno) and all the rebuilt houses are guaranteed customers.

Systemic effects like prolonged loss of power or water have the greatest impact, and are also the hardest to plan for. It's really hard to work out all the consequences of unplugging the electric grid in Mountain View for a week, let alone when there's multiple other emergencies happening in parallel.

Maybe it's not a great idea, but it wasn't someone's "idea" or conscious decision, so there's not much you can do about it. Good luck convincing millions of people in the Bay Area to relocate to the Australian outback.

Try rebranding it as a year long Burning Man and see how they flock by the thousands.

I'm pretty sure one of the big enjoyments of such outings as Burning Man is an ability to come back to civilization as soon as it's done.

Does anyone have data centers in Silicon Valley? Real-estate (and electricity and cooling) costs are amazingly high, so it's not that good a place for a datacenter.

http://www.datacentermap.com/usa/california/

There's 34 listed just for San Jose and Santa Clara...

There are tons all around the South Bay. And of course corporate HQs.

"Destroyed" is an overly dramatic word, and wrong.

Earthquakes don't destroy digital infrastructure. At most they might dislodge a few things temporarily.

Historically you might note that fires have sometimes occurred as a result of earthquakes, but then it's unlikely those fires were in data centers.

Stuff does sway around a bit, but it's not a huge deal for the digital infrastructure.

Earthquakes destroy water and power grids, and transmission lines, which is a big deal for the digital infrastructure.

It's one reason (the other being cost) we moved out of a datacenter located in San Francisco's SoMa district.

Your better off spreading the risk all over the place. If all the places go down at the same time, I think you have bigger problems. Besides, I think most of the large institutions have accounted for this already.

I think the real risk here is the loss of life (bus factor).

Here are some useful maps:

http://www.hazardscaucus.org/presidentialdisasters65-98.JPG

http://www.datafoundry.com/wp-content/uploads/2011/04/natura...

Which portion of the world's digital infrastructure is in the bay area? Where the companies are located doesn't really have that much to do with where their servers are. Maybe to some extent, but less than you might think.

Also, if it helps you sleep better at night, none of the root dns servers are in the bay area: http://public-root.com/root-server-locations.htm

Agreeing with you on not putting everything in the same place. In general you'd be ill advised to put anything in one place. Hedging is the way to go. There is no reason why centers of the digital age couldn't be around the world in say SF, New York, Beijing, London, Berlin, Singapore, Sydney or so. Not sure I'd still put everything in one place like the outback, far away from civilisation.

Perhaps the people down-voting you might like to share _why_ they think you said something wrong.

FWIW, I would tend to agree - in general terms.

I didn't downvote, but it seems that all these companies would(should) have backups outside of the bay area. I can't imagine major services going down for an extended period of time. I do foresee a significant drop in overall productivity in the area for a longer period though.

> I do foresee a significant drop in overall productivity in the area for a longer period though.

True, though there are a lot of people working in that area. Deaths at the wrong time could be catastrophic.

For one thing Australia is pretty far from everybody.

Also, while a lot of companies are in the SV, I don't think many servers are actually there. Google wouldn't disappear from an earthquake.

An earthquake is not an endgame either. Sendai and Kobe are still on the map. Though I don't know how prepared SF is for this.

I'd say it's a lot less bad than putting a nuclear power plant square on top of a fault line. Yes, cooling water is nice, but they build diablo canyon in a when-not-if there's an earthquake area, and then discovered it was smack on tip of a fault later.

Because you'll be swapping the issue of death-by-earthquake with death-by-enough-heat-to-melt-silicon-PCBs.

Why not build it in some frozen area instead? All the same advantages and your cooling costs are ridiculously cheaper.

Put it next to the Svalbard seed vault in Norway, http://www.atlasobscura.com/places/svalbard-seed-bank ?

Ok for the general area (Alaska, north of Scandinavia, etc) for stuff that needs chilling and stable geology/politics.

But don't put anything close to the seed bank, which a crazy with a nuke (IS, Putin, Iranian ayatollahs, etc) might want to target...

Edit: Maybe not Alaska for geological stability.

To what infrastructure do you refer, besides people?

Each individual company can probably use the time and money to mitigate against much largar risks to its individual business.

(I did not downvote, btw)

Even more reason for a distibuted workforce that can work from home.

plus, if in the middle of the Australian outback, the datacenters can serve as the plot device for a 5th Mad Max film.

No one put it there. It evolved there.

Your point was diluted, the conversation misdirected, when you used Australia as the alternative. Of course there are countless options that remain in the continental United States and have far less of a risk profile. Nevada, for instance.

Ultimately the coast of California is getting set up to be the biggest case of hubris and short-term thinking in human history: Where we knew for decades what the significant risks were, with ever increasing accuracy and sureness, but kept going ahead anyways. If such a major event eventually happens, as every seismic model predicts, I don't think people will say "Well, we knew this was coming.", but instead will point fingers, question why the government didn't prohibit building up there, etc.

Reply on news.ycombinator.com