Hacker News Reader

Comment by pmoriarty | Hacker News Reader

I often wonder what's going to happen to all the great information that's on sites like HN, Reddit, Stack Overflow, or other popular forums when the companies that run them finally bite the dust or stop being interested in running these websites. It would be a real shame to lose all the advice, howtos, solutions, opinions, conversations, etc.

I'm hoping these sites have archives, and that those archives would be made publicly available in case of such a disaster. But maybe even then they'll be buried behind a shitty web interface such as Google's web interface to old Usenet archives. Or maybe they'll just disappear completely, which would be a huge waste.

View on HN

http://www.archiveteam.org/ is specifically dedicated to archiving web services before they go down.

" Yahoo! Answers: It's Yahoo. It's been acquired."

That's the kind of common sense and succinctness I like to see. It's Yahoo so mismanagement or disappearance is a high risk. It's been acquired. Assets often get gutted or shelved after that. Two statements in five words tell you everything you need to know about why Yahoo Answers might need a mirror. :)

Since the days of stone tablets, the accessibility of information has only increased. But its volatility has also only increased.

On the flip side, a lot of the "information" on Reddit & al is just junk, about same value as pure noise.

You might be interested in https://archive.org/.

Does it actually archive all of HN, Reddit, and Stack Overflow (and related sites)?

I've seen lots of omissions in archive.org. Not that it's not great. I love it. But there's a lot more that needs to be archived than what's in there.

The Archive Team[1] tries to scrape such records when the sites are in danger of closing down. Currently my Warrior[2] is pulling pictures from Panoramio[3]. Unfortunately, resources are often scarce, particularly storage.

[1] http://www.archiveteam.org/index.php?title=Main_Page

[2] http://www.archiveteam.org/index.php?title=ArchiveTeam_Warri...

[3] https://news.ycombinator.com/item?id=12732692

I believe they're currently preparing on working on Vine, and, from what they wrote on Twitter just today (can't search for it right now, don't have a stable connection right now), they don't have any problems with the storage at the moment thanks to Archive.org.

I suppose we could email them with a request to make sure they scrape everything off of first, few pages of HN or key subreddits. I'm not sure about Stack Overflow where they or Google would know best based on hits certain questions get which are best to archive. Maybe just archive whole thing. Sites might even pay them to as a high-availability or backup option. Low odds there but asking doesn't hurt. ;)

The Internet Archive (and Stack Overflow) are way ahead of you here. SO is working directly with the Archive to publish anonymized database dumps.

https://archive.org/details/stackexchange

That's awesome. Thanks for the link!

Similarly, you might be interested in the "Save Page As..." feature of web browsers along with the bookmarking one. I too was worried the best stuff would go away. Plus, Archive.org is sort of a single point of failure, isn't it? So, I just started copying best comments into text files or bookmarking their info if they suited my purposes. Could straight up save the whole thread. If it ever goes down, we upload our collections to one or more repositories in a similar format.

Note: I'm ignoring the complexity that copyright law would introduce into this to just focus on keeping the key info.

I do try to do my own archiving of things I'm interested in. But it's a lot of work, I don't get nearly everything I'm interested in, and I can't predict what I'll be interested in tomorrow, or what I'll need then. Not to mention there's all the things everyone else is interested in.

The best thing really is to do a wholesale archive of the whole site, and keep doing so at regular intervals. Just like archive.org is doing (at least for the sites it archives).

The moment that copywrong stops us from keeping a full and accurate record of history, it's time to eject the broken and destructive system entirely.

Sounds nice but masses dont care that much.

Printed copies stored in libraries of UN and each of the Security Council permanent member countries, and whoever volunteers to make it "technology proof". Further, the paper and ink could be engineered to last for many centuries.

Reply on news.ycombinator.com