Hacker News Reader

Comment by AndrewStephens | Hacker News Reader

The is highly unfortunate and a bone-headed move by Microsoft. But it does highlight two points that often get missed now that we rely on the web.

1 - If you don't host your own content then you have no control over it. Microsoft deleted all their blogs. YouTube and Facebook will one day cease to exist. This comment and all of yours will disappear once news.ycombinator.com is defunct. If you want something to last, host it yourself or be prepared to recreate it somewhere else every few years.

2 - Once you publish something at a URL you control (see point 1), you have a responsibility to keep it available at the same URL. Nobody is going to hold you to it but you will break 3rd party sites that link to you. Microsoft really dropped the ball here.

View on HN

The Internet Archive is doing some ridiculously important preservation work that we all just take for granted.

Are they? I mean, just recently there was a post on HN asking whether the G+ backups were necessary in the sense of creating a permanent record of everything people submit online.

Make no mistake, certain content is worth archiving, but imagine if we'd been archiving everything since our species first days.

Now we have the capability, we're obsessed that we must archive everything; how useful are the older IMDB comments really?

I dread to think how much storage is 'wasted' in these archives storing useless content, and how useful it really is to people in reality.

There are archives of every reddit comment ever made, but there are many hundreds of thousands of spam or useless comments....

I don't think the Internet needs an immutable record of every piece of content or every hyper link to preserved for the rest of time, we need to discern what's really important and throw away the rest, if not to make it easier to parse (but, since that's difficult, we just grab it all which whilst great presents its own issues in terms of required resources).

I think it would have been immensely valuable for anthropological reasons to see the mundane details of the lives of people who lived thousands of years ago. If we just keep only the most important works it doesn't really give a clear picture of how people actually lived at that time. That said, I definitely agree that there will be diminishing returns if you keep everything.

imagine if we'd been archiving everything since our species first days.

Then there would be no "Mysteries of...!" shows on basic cable.

At the very least it will be useful to future history/sociology phd students who will be able to pick any moderately active forum or subcommunity from the past and write a dissertation on its idiosyncracies

Today I can enjoy reading several thousand years old books like Herodotus travels.

Within the same distance into the future, most of today's knowledge will be gone.

To be fair most of the books written in Herodotus’ time are also gone! Many of them we only know about from Byzantine library catalogs or summaries, but surely many more have just completely vanished from knowledge or existence.

With the existence of things like the Internet Archive, it’s quite possible the public web content of today will be well preserved for centuries to come.

Right now the Internet Archive and BitKeepers feel like Alexandria's library, a kind of single point of failure with a big loss to all of us if something happens.

And we also need to keep running the applications that know how to deal with specific file formats.

But it's much easier to mirror the Internet Archive than it was to mirror Alexandria's library. In fact, it already has one official mirror (which is hosted by the modern library of Alexandria!).

FWIW, with today's typical Internet speeds and storage prices, it's entirely feasible for individuals to mirror the entirety of Wikipedia and Project Gutenberg (done both myself, and a bunch of other stuff like TED talks). Throw in a mirror of the entire Debian source package repository, and there's your apps, too. How many copies are already around? Who knows, but I'd say we already have a lot more redundancy than at any point in the past.

Of course, IA is to any of these like these are to what you could store locally circa 90s. I don't think we're going to be at the point where an individual can easily store 40 petabytes of data anytime soon. But for large entities - governments and corporations - this is feasible. And I suspect there are some private full IA mirrors out there already, in places like NSA's Utah data center.

And then you can make distributed backups, too: http://iabak.archiveteam.org/

Thanks for the heads up.

Yeah I guess we need to trust that some of us will keep our recipe lists alive. :)

To be fair, if we do manage to preserve the Internet Archive to a significant degree, information on html and other filetype standards should be readily available for future recreation.

Within the same distance into the future, most of today's knowledge will be gone.

A bunch of it is already gone.

Think of all the music recordings that didn't make the transition from 78's → LP's → CD's → streaming.

Think of all the movies that didn't make the transition from film → videotape → DVD → streaming.

I'm always sad when I see the pile of books being discarded by the Chicago Public Library for 10¢ a piece, knowing that those that aren't picked up by the public end up in the trash. The information in them may be outdated, but it's an important window into what people felt/thought/believed in a particular period of time.

The world loses a metric assload of its history each day.

Most of the knowledge from back then is also lost. It wasn’t written down or if it was, most of that was lost.

True, but paper and clay/stone tablets happen to be more resistant than magnetic storage and continuous change of application specific formats.

I presume that laughing at idiots who actually wrote this sort of thing without realising the irony will be one of the periodic amusements of next century.

The only way to increase the information's chance to survive any significant length of time is copies. But making copies of digital artefacts is so trivial we made it totally routine, when did your backups last run? Whereas historically objects like tablets were only reproduced if there was extraordinary interest, so we have mostly Bibles, and random lists that survived entirely by chance due to just how many there were, often inventories or other boring records.

Well there are plenty of surviving tablets with shopping lists. :)

I doubt you're ready the original manuscript though. You're reading copies of copies, there is no reason why content on the Internet will not continue to be copied in the same way.

It's a matter of scale. You can only make a copy of the whole internet by committing to that purpose, and there's only one outfit that does it in a public manner. I can't imagine what it would take to duplicate that database right now, and it increases every day.

They already have one official mirror.

http://web.archive.bibalex.org/web/

In comparison, imagine the amount of effort and man-hours required to copy a similar amount of printed works distributed across the planet.

It is much easier to copy paper and tablets than our snowflake storage formats.

Reply on news.ycombinator.com