Hacker News Reader

Comment by oceanplexian | Hacker News Reader

It’s weird that we are in the minority (I guess I’m not shocked since this is HN). Very little on the Internet is useful for survival in an Internet down scenario. And if the Internet is gone, we must have far bigger problems. Libraries are going to be far more useful, and books don’t require electricity. I grew up in a place with lots of power outages and the main thing you are worried about is having hot water for a bath or a being able to cook a meal.

Tangential thought but we should probably work hard to preserve libraries in the future. Real ones, with books. They are really unmatched when it comes to longevity and safeguarding information in a way that computers cannot replicate.

View on HN

I can't get behind these restricted whatif scenarios. If the internet were to disappear there would be ensuing outages of critical services and shortages of essential items very quickly. (In part because of the inevitable mass panic and hoarding).

If I knew the internet was going out tomorrow I wouldn't spend any time on the internet at all. I would go the grocery store, gas station, friends houses and then get as far from major cities as I could.

Thought experiments are a springboard for an area of thought, not necessarily a literal question to be answered. When someone asks you "what would you do with a billion dollars" responses like "but I don't have a billion dollars" or "nothing, I'd be investigating how I got it" completely miss the point. It's not about whether the scenario would realistically play out it's about setting the stage for certain types of thoughts without prescribing an exact question on everyone. Maybe you'll never realistically be at a train track with a fork in a road, 1 in the alternate path and 5 in the active path, with nothing more than the option to flick the switch and no other consideration to make... but it sets the stage for interesting thoughts to consider and talk about.

It can lead to much more varied and interesting discussion that direct questions, if you're willing to get over the non-literalness.

Yes I know what a hypothetical question is... this question presumes that downloading content is something I would do in this scenario.

Like asking: "If you could meet one celebrity in person which Kardashian would it be?"

> They are really unmatched when it comes to longevity and safeguarding information in a way that computers cannot replicate.

How so? There are long-term digital storage technologies that would long outlast any book and are many orders of magnitude denser.

The only requirements for reading a physical book is that you know how to read and can turn pages.

If it's digital storage, you have to have electricity, a compatible device, an understanding of the storage, and software that can read it.

> If it's digital storage, you have to have electricity, a compatible device, an understanding of the storage, and software that can read it.

And, increasingly, DRM servers that will allow you to read it.

No there is not. What do you mean!

Nothing has been verified to work beyond 50 years, and those with data errors and failure rates.

There are those CDs made out of rock, but they have never veen proven to pass the test of time.

> There are those CDs made out of rock, but they have never veen proven to pass the test of time.

You're saying that something that has existed for less than 50 years doesn't count because we haven't been able to actually test it for more than 50 years, even though we understand the physics behind it and can theoretically predict how long it will last...

M-disc? I'd struggle to get hold of a blu-ray disk player to read one today let alone in 50 years time.

And a quick google reveals a lot of people are very worried about counterfeit disks too.

While I think common digital media outlasting analog forms in terms of integrity over long periods of time is unrealistic I do have 40 year old CDs from 1984 that are still bit perfect as of just a couple years ago (verified against online checksum databases for the same releases), so it'll be interesting seeing how long they last.

Pressed CDs are pretty good in terms of durability, but how are you gonna get one produced in a single day? (Per the prompt, the Internet disappears tomorrow, not in a couple of weeks.)

There was a period of time where pressed CDs were manufactured poorly, with the aluminum layer inside exposed to the outside, resulting in corrosion over time and loss of readability on those CDs.

Overall, though, properly-made CDs, handled carefully, have been excellent at storing data long-term.

But while this is nice enough I guess for storing individual musical albums long-term, it's not practical for storing truly large volumes of arbitrary data. CD-Rs and CD-RWs have not had the same durability demonstrated at all (quite the opposite in fact). DVDs are better at almost 4GB per disc, but here again only the factory ones are actually durable, and 4GB isn't going to store much these days, perhaps one movie with high lossy compression.

While my comment wasn't about the feasibility of pressed CDs for a mass blackout event but just an example of long-term integrity of existing digital media, it's unfortunate that a forum (MyCE) dedicated to tracking integrity of user-writable optical discs unexpectedly closed a couple years ago due to the webmaster pulling the plug.

It had users who carefully performed benchmarks on media for more than a decade to see which types and makes held up best over time, along with best practices. Few have the interest or patience for such things so it's unfortunate to just have such info vanish.

I will add though that what's missing from the discussion is Blu-Ray, which allows up to 128GB per disc. (I only vaguely recall reading some critique of BD DL discs so can't say how it might compare long term though, apart from the greater cost at such capacities.)

Somehow it seems ironic that a forum dedicated to understanding the long-term viability of data storage, an important topic lately because of the unreliability of 3rd-party providers (like cloud companies), itself became a victim of the unreliability of its own webmaster.

128GB BD-R discs do exist, but at $219 on Amazon for 25 discs, that's about $0.07/GB. It would be MUCH cheaper to just buy a stack of refurbished enterprise-class HDDs and store your data on those, in triplicate, with a filesystem that has error correction (like ZFS). Personally, I would bet on HDDs used this way still being readable and not having bit-rot after 50 years over 4-layer BD-R discs.

Wouldn't it be more like $0.07/GB for BDXL? If one got particularly lucky with HDD failure rates perhaps they'd survive running that long in RAID but one would expect some replacements over such a long period.

Some other things to consider are at high capacities all HDDs use helium now, which slowly leaks (WD/HGST have a SMART stat about the level*) and the cost of running drives/associated computers/maintenance over a long span vs the up front cost of passive writable media (edit: for some reason I assumed this was what was meant but they could be left cold which would likely increase survival odds and be cheaper).

* And there isn't much long term data about it that I could find, though some have reported between 1-5 years the SMART stat either remaining at max or dropping a few digits. Even Backblaze outside of their first article a year into using them hasn't seemingly continued reporting on the stat that I've noticed. I get the sense though that other types of failures are expected sooner than leaks.

>Wouldn't it be more like $0.07/GB for BDXL?

Whoops, thanks for pointing out the math error; I've fixed it.

>If one got particularly lucky with HDD failure rates perhaps they'd survive running that long in RAID but one would expect some replacements over such a long period.

I don't think so: I'm not talking about keeping these drives spinning for 50 years, but rather in cold storage, just as we'd do with the BDXL discs.

>Some other things to consider are at high capacities all HDDs use helium now, which slowly leaks (WD/HGST have a SMART stat about the level*) and the cost of running drives/associated computers/maintenance over a long span vs the up front cost of passive writable media.

Helium leakage is an issue I didn't think of, and I don't know how sitting in cold storage for 50 years would affect this. But again, the costs of running drives/maintenance/etc. should be zero, because I'm comparing apples to apples. No one would seriously propose a massive array of BDXL drives with BDXL discs continuously available, so likewise I'm proposing just keeping 10+TB HDDs in cold storage.

What technologies are you thinking of?

Ubiquitous access to reader devices is also a factor, and I can’t actually think of anything that fits that bill.

The best one seems to be https://en.wikipedia.org/wiki/5D_optical_data_storage but there's also M-DISC and DNA storage. Microfilm also lasts about 500 years.

As for ubiquitous access, store a reading device or instructions on how to build one along with the data. If you're unable to do that, then I doubt you would be able to keep a massive library of books around for very long either.

There's also no financial incentive to build technologies like this. If the world actually got together and tried to build long-term digital storage then I'm sure we could come up with something even better.

AFAIK a library doesn't really require maintenance, unless there are extreme weather conditions, the books will survive for a long time on their own. It's only the ancient books that require a controlled environment, because they already lasted for centuries and we're trying to have them last for even more due to their historical value. So you would be able to keep libraries around for long in many (most?) scenarios. Instead, the devices you need to read those storage media require high-tech factories to be manufactured. Just having the instructions to build one will not suffice.

How long? I don't think a library would last more than 100,000 years given natural disasters and plate tectonics etc. All you need to do is make a reading device that can last for a similar amount of time. And if the device itself wouldn't last that long then you could provide as much long-lasting equipment or material as possible to help build it.

The scenario you're describing is incredibly specific. It requires a post-apocalyptic world where humans have survived, but have somehow completely lost all ability to access past knowledge. Civilization must be advanced enough to access and read a library that has been shielded from the elements for millennia, but not advanced enough to build microscopes or lasers, even when given precise instructions on how to do so. It must be far enough into the future that any possible small high-tech reading device we could create is unlikely to have survived, but not so far into the future that a very large library structure is likely to have collapsed.

> I don't think a library would last more than 100,000 years given natural disasters and plate tectonics etc.

100000 years is a very long time. And in that time, you have good chances of reeboting civilization and reconstructing our current industrial world.

> All you need to do is make a reading device that can last for a similar amount of time

Easier said than done, and why would you need to do it, if libraries already solve the problem?

BTW I think we're considering two different scenarios. Libraries are excellent at solving the scenario given here, i.e. the internet collapses tomorrow.

> https://en.wikipedia.org/wiki/5D_optical_data_storage

An interesting technology, but also not exactly something I could get at my local Best Buy today.

M-DISC, assuming it's writable using consumer Blu-Ray writers, is promising though – Blu-Ray drives can probably still considered ubiquitous enough in a pinch.

> DNA storage

DNA is in fact extremely unstable unless it's part of living organisms that constantly error-correct and replicate it, and even then you have random mutations.

Information has outlived entire civilizations because of books. The key is the technology needed to decode and read it, which is just humans themselves. Either people still exist who can read and speak the language or closely related languages, and if not, we can hope to find something like a Rosetta stone or use statistical analysis that relies up on the commonality of all human languages.

Any digital storage device is simply giving you a bit stream. Being able to read the bits at all might rely upon technology that no longer exists. You need to know the layout of the medium, where to start reading, how to perform any built-in error correcting, what constitutes data versus metadata. Once you read the bits, you still need to do all of that again, but this time at the level of the filesystem. Then you need to do it a third time at the level of the file format. Then you get, at best, something like a consecutive sequence of unicode code points. Now you still need to know unicode.

We have no idea if these sorts of technologies will be remembered in 3,000 years, but given the history, there's a very good chance people will still be able to read Sanskrit and Latin, and the way the human eyeball accepts and decodes light waves will not change.

I think looking at history is a terrible way to make predictions about the future. The world will never again be anything close to what it was in the past.

You think assuming humans in the future won't have eyeballs capable of viewing visible light is a bad assumption?

If the humans of the future are all blind, I think we can forget about worrying about preserving civilization.

The Lindy Effect[0] agree with you.

0: https://en.wikipedia.org/wiki/Lindy_effect

There are many, many libraries that have sections that have an almost military-level protection (protected atmosphere, security, and so on) I think humanity has done a good job in general on this front

Yeah, and where exactly are those libraries? Are they in safe places that won't be obliterated if a war breaks out? I don't think so; they're mostly in the most likely to be targeted locations.

If humans were serious about protecting knowledge, they'd put these libraries on Svalbard or in Antarctica, or better yet in a lava tube on the Moon.

Reply on news.ycombinator.com