Comment by livrem | Hacker News Reader

Comment by livrem | Hacker News Reader

They forgot one: Natural disaster deleted all the data and there was no backup.

I was a paying subscriber to a great little site called magweb ~20 years ago. They scanned old and new (military) history and (war) game magazines and posted HTML versions, with permission of original publishers. It was really nice and explicitly allowed users to print or save copies of articles. No DRM. Perfect example of how a site for reading magazines online should work. Everything just basic HTML that even worked great to read in phone browsers 20 years ago.

Then Hurricane Katrina hit and the server that was apparently running out of the owner's basement somewhere in that area was flooded, with no working backups. I still have not found any traces of most of the 40000+ articles, other than the few I had saved, that used to be on that site. Since it was paywalled only a tiny part of the site is available on the wayback machine.

https://web.archive.org/web/20050529083811/http://www.magweb...

(No, I can't imagine running a business and scanning magazines for 9+ years and not make sure to have backups of everything.)

There was also the OVHcloud data centre fire in 2021, although it's debatable if it'd fall under the same "natural disaster" category.

https://www.datacenterdynamics.com/en/news/ovhcloud-ordered-...

I know at least one startup that closed its doors because they lost both production and backup data on that OVH data center.

And that's why 3-2-1 specifies to have one _offsite_ backup

Preferably with a completely different provider, in case something takes out there entire network somehow (or they go out of business).

But make sure you check where the other provider is hosted: I read it one user (individual, not a startup or other company) who had backups of stuff on a shared host with a second shared host. It turned out that they were both running on servers in that one OVH DC and they were cheap hosts with no better backup/dr plan themselves...

https://xkcd.com/908/

I'm gonna Craigslist all your stuff! https://xkcd.com/1150/

I personally was affected by this fire, although I've always kept 3 month backups of production data, encrypted, on-site, just in case of emergencies like this. Haven't touched their services for anything production related ever since

Similarly: Terrorist attack. After 9/11 I remember that some websites for WTC-based companies went offline, presumably because they were hosted on computers in the building.

Rather infamously, NYC's Office of Emergency Management was located in 7 WTC, and had critical communications facilities (broadcast tower) atop 1 WTC:

Since 1999, New York City’s Office of Emergency Management, charged with coordinating all aspects of the response, had occupied permanent headquarters in Seven World Trade Center, on Greenwich Street, just north of the landmark twin towers. A vital communications link was the radio repeater system based on the ground floor of One World Trade Center, the north tower. The loss of those facilities – and key personnel working there – significantly hampered the response.

<https://theconversation.com/disaster-communications-lessons-...>

<https://www.computerworld.com/article/2510996/9-11--top-less...>

I seem to recall an instance (I thought it was the NYC OEM, may have been another) in which both primary and secondary data archives were both within the WTC complex. Best practices now are to locate secondary / backup services at least 100--200 km from the primary. Preferably within different watersheds, seismic regions, etc.

Widespread disasters are comparatively rare, but can affect considerable areas, and locations sufficiently proximate might well be affected.

Reply on news.ycombinator.com