Hacker News Reader

Comment by tkgally | Hacker News Reader

The linked article mentions “There will also be servers in the building.” Here’s a bit more information from the invitation I received last week to the Internet Archive’s event marking the launch of their new Canadian headquarters:

“The Canadian Data Center is a secure and sustainable back-up for the entire Archive. Our goal is to continue providing access to knowledge for decades to come, and this ensures we can and will. This Data Center stores everything in our Archive – as a back-up. It’s a full, second live copy preserved outside the US — which ensures our data cannot be compromised by threats to a single system or location.”

The Internet Archive is a nonprofit charity and welcomes donations:

https://archive.org/donate/

View on HN

Canada is an interesting choice as a backup location for when the USA becomes unsuitable. Most people will use USA infra to access it, and any sort of war scenario that threatens mainland USA... I don't see Canada being not at risk. Heck, South America would have been a better choice for spreading out this huge and imo important archive (it's only a disaster copy after all, I'm assuming there are the normal backups/redundancies against disk failure and fire).

This sort of thing, "hey we're opening a tech hub" and "hey we're storing our disaster copy in a very closely related economy a few km away from the current country", make me wonder if I should instead start my own copy and try to get funding to secure e.g. one recent snapshot of every website as a first goal, rather than donate money to the IA directly (as I have multiple times in the past).

There is also one in Alexandria (Egypt).

But yes i think there should be something like this:

https://wiki.archiveteam.org/index.php/INTERNETARCHIVE.BAK

I can't understand why there isn't inertia behind restarting ia.bak or similar efforts. That seems critical to me, way more so than an official mirror which would be vulnerable to the same takedown and other pressure.

Because it’s a lot of work.

I'd also guess because (potential) mechanisms to do so would be attacked on forums like HN because they sound too close to crypto/web3 and most people wouldn't bother to separate ideology from technology and create a PR nightmare for internet archive. So it's probably harder to recruit people to work on it.

See Filecoin Foundation and their work with IA.

Just do something similar to folding coin, which is used for things like protein folding. Perhaps an Internet archive coin that can be earned by hoarding backups of the archive on hard drives.

Its also a really hard issue. My fav is IPFS, Its the most common suggestion but it only works on paper. IPFS Allows people to pin things, but it still has to be stored /somewhere/ and, it gets really damn slow once you throw more then a million files at it (I know... we tried)

Overall, Its just a giant archive, its hard to store that much data in a cheap manner that will last 20 years.

I can understand why.

Around 1 PB of the Archive is available via torrent for backup. The full Archive was around 10 PB back in 2012 so I imagine it's quite a bit bigger, now.

https://help.archive.org/help/archive-bittorrents/

North of 40PB nowadays. You can find these stats buried in their public links. They do not hold up well to traffic so I will not link them.

Our numbers as of the end of 2021: https://archive.org/web/petabox.php

    4 data centers, 745 nodes, 28,000 spinning disks
    Wayback Machine: 57 PetaBytes
    Books/Music/Video Collections: 42 PetaBytes
    Unique data: 99 PetaBytes
    Total used storage: 212 PetaBytes

Geeze, Its much higher then I though! 212PB is damn hard to store!

Any war scenario that threatens the mainland United States would probably involve nukes starting flying, and you'd probably be screwed regardless of where you put it. I'd be more concerned with an asteroid that hits just right.

"a few km" seems like a weird way to put it, as a "few" is typically 2 or 3. And just specifying "US" and "Canada" is weird, as an archive in the US and Canada, could be separated by 10000km without any effort.

That said, these archives are close, but I beleive the real reason for this, are things such as take down, erasure, and other court requests in the US, would not apply in Canada.

This is doubly true, if you don't serve data to US citizens from Canada.

So it allows for easy physical maintenance trips, a second off-site backup (weather, rioting, looting, earthquake, etc) unlikely to take-out both.

I also think South American countries are far more likely to descend into rioting, violent takovers, coups by armies/etc, than any place in North America.

I wouldn't assume anything with our current (Canadian) government, especially if the US government applied strong leverage to try to get material removed. Even looking at recent history with (purported) terrorists, our record at resisting US pressure is not good.

Putting it in Vancouver is also not the greatest idea when you factor in earthquake risk, although it's far enough away from San Francisco as to provide a reasonable backup. I'm sure there are other factors that come into it though, and I'm sure it makes sense on the whole.

I wouldn't just say "current" - Canadian governments have been pretty subservient (for obvious reasons) to the US since WWII, at least.

What a weird subthread, with made up, unsubstantiated claims.

It also shows a shocking lack of understanding. The "Canadian Government", being the executive branch in this case, has no bearing on this. US requests to seize assets, force shutdowns would be handled by Canadian courts, the judicial branch.

Which do not kowtow to anyone politically, US or otherwise, and never, ever have.

oddly though - in some science and software themes, Canada is more closely aligned to the EU and Commonwealth than the USA. (American here..)

> So it allows for easy physical maintenance trips, a second off-site backup (weather, rioting, looting, earthquake, etc) unlikely to take-out both.

> I also think South American countries are far more likely to descend into rioting, violent takovers, coups by armies/etc, than any place in North America.

North America is constantly hit by various disasters, from earthquakes along the ring of fire to hurricans on the south and east coasts, to ridiculous cold weather in the continents. For more manmade disasters, George Floyd aftermath and Jan 6th 2021 would tend to disagree.

Switzerland would be a good place for an archive. Connected, stable, neutral

> For more manmade disasters, George Floyd aftermath and Jan 6th 2021 would tend to disagree. Switzerland would be a good place for an archive. Connected, stable, neutral

The threat of a Nazi invasion of Switzerland was a more severe threat to civil order in Switzerland than any protest or riot in America or Canada in the past century.

> Nazi Germany repeatedly violated Swiss airspace. During the Battle of France in 1940, German aircraft violated Swiss airspace at least 197 times.[17] In several air incidents, the Swiss shot down 11 Luftwaffe aircraft between 10 May and 17 June 1940, while suffering the loss of three of their own aircraft.[17] Germany protested diplomatically on 5 June and with a second note on 19 June which contained explicit threats. Hitler was especially furious when he saw that German equipment was used to shoot down German pilots. He said they would respond "in another manner".[17] On 20 June, the Swiss air force was ordered to stop intercepting planes violating Swiss airspace. Swiss fighters began instead to force intruding aircraft to land at Swiss airfields. Anti-aircraft units still operated. Later, Hitler and Hermann Göring sent saboteurs to destroy Swiss airfields but they were captured by Swiss troops before they could cause any damage.[18] Skirmishes between German and Swiss troops took place on the northern border of Switzerland throughout the war.[citation needed]

I think any war scenario that threatens the mainland US probably threatens the rest of the Western World, and probably the entire developed world too.

And in any case, there are plenty of scenarios where Japan, or Germany, or India are threatened, but vanishingly few where North America is.

I'm pretty sure if the US mainland is threatened by any foreign actor, the entire planet is toast. They could maybe have a cold archive in Svalbard for future alien xenoarcheologists trying to figure out why human civilization self-destructed.

Now, a civil war, that's a different story.

> Canada is an interesting choice as a backup location for when the USA becomes unsuitable.

Good for a World War III scenario. Less good for a "delete this [wide class of information] or else" scenario, as there are now two governments to appease and the Canadian one is far less free-speech-friendly than the US one.

Not sure free-speech-friendliness is really relevant here, since the most likely avenues of attack would be copyright or it being classified information. For both those things the US isn't exactly the best candidate either.

I consider at least the latter to be part of free speech as well. Then there is privacy (the Wayback Machine is likely a C&D magnet here), pornography/obscenity (many places are stricter than the US) and "hate speech" (here the US is unbeaten). Besides that, having the data spread over two countries means being susceptible to both governments.

This is probably to protect the archive from the US.

In a war that threatens mainland USA, nowhere on Earth would be safe. Perhaps in a nuclear bunker in New Zealand, or on board the ISS.

> Canada is an interesting choice as a backup location for when the USA becomes unsuitable. Most people will use USA infra to access it, and any sort of war scenario that threatens mainland USA...

Honestly, the Internet Archive is probably toast wherever it is if there's "any sort of war scenario that threatens mainland USA."

I don't think we should kid ourselves that something like the Internet Archive will be able to be preserved over the long term. It's a kind of thing that's bug, fragile, and expensive to maintain. Most of that stuff is destined to be lost eventually, probably sooner than we'd expect.

I think the risk scenarios in question are more legal/political than war-related.

As others have noted, the set of safe locations to store an enormous, expensive, extremely fragile collection of uncensored material during a world war is approximately the null set.

Not only is it close, but it's in an area that could have earthquake. I think we need to bury the servers in Manitoba for extra security.

I was at the building recently for an event, and I peeked into the 'office' and there were servers. Like 8 of them. There's absolutely no way they're going to be mirroring data in this location. They might establish another location within Canada for doing backup but if you're serious about DR having something in Vancouver isn't exactly the best idea. Most companies would choose to locate in the interior of BC around Kelowna / Kamloops region.

I think they are preparing for the wave of christofacism being lead by the current GOP leadership that seems to be growing inside of the USA.

Seems odd to go to all this effort of making an entire second copy, only to put it just over the border into a country with not too dissimilar laws. As another commenter said, would've been better to go with Iceland or something.

A backup of the archive has been hosted at Bibliotheca Alexandria in Egypt since 2002 [0] and is accessible at [1]. I'm not sure what the state of that project is but I guess it's not a "live" copy, the first link here describes an incremental upgrade over the years.

Agreed that Iceland would be a nice international headquarters.

[0] http://www.bibalex.org/en/project/details?documentid=283

[1] http://web.archive.bibalex.org/web/

I will note that Alexandria backup has gotten a tad out of sync. No fault of theirs, the data set just got too large to manage.

Reply on news.ycombinator.com