Music Industry's 1990s Hard Drives Are Dying | Hacker News Reader

Music Industry's 1990s Hard Drives Are Dying

arstechnica.com

49 pointsm46342 comments

[dupe]

Lots more discussion on the source, as referenced in the article: https://news.ycombinator.com/item?id=41504331

So here’s a question: let’s say I store a backup in AWS Glacier.

Do they just store it and forget it till I try to access, or copy it to new media every month/year to make sure it’s still accessible?

Same question for Backblaze B2 and such.

Does anyone know?

"Durability" is the term you're looking for. S3 and Glacier famously have 99.999999999% durability. The quote goes "if you store 10,000,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000 years."

Backblaze makes similar claims.

I think the commenter was asking if they refresh the storage enough for it to not die if natural causes. If data is left around on most storage media it will be gone within 75 years if not refreshed somehow.

Exactly..

This is only relevant if you have billions of objects in S3, when your failure rate can be amortized. Otherwise either Amazon is going to go bankrupt or you’ll be the one person who has total data loss by chance.

Already happened didn’t it?

It’s M-of-N erasure coded and periodically verified.

So yes, they copy it to new media, after it has failed.

Math is crazy.

Glacier might also have a heuristic for media lifetime, but ehhh.

Counterquestion: How much money do they lose when you notice in 50 years that your data are gone? And how much money you payd until then?

So, it's just simple math that you can use to calculate your answer.

You completely missed the intent behind my question.

I’m trying to figure out an appropriate strategy for backing up things that are important to me.

If they move things around, great, else I’ll need to do it myself.

AWS does regular integrity checks on all media, and repairs any errors it finds. This includes Glacier and Deep Glacier. (It's in the first few bits of the S3 docs)

Backblaze B2 is untrustworthy. I specified a bad encryption key- and it accepted the data, but it wasn't retreiveable.

For the extra $1/TB/mo Wasabi is the best For hot backups, IMHO.

Also, I theorize that you can avoid the major cost of AWS Glacier deep archive by downloading it through CloudFront- which has 1TB/no free.

So: 1. Make your restore request near the end of the month. 2. Glacier restore ~2TB of days (bulk restore, slow.) to S3 hot tier. 3. Download 1TB while on the tail end of the first month, and another 1TB when CloudFront resets.

The huge egress costs are minimized this way, as well as minimizing the GB/s charges of regular S3.

> Backblaze B2 is untrustworthy. I specified a bad encryption key- and it accepted the data, but it wasn't retreiveable.

That’s an odd dig at B2. If you had done the bare minimum to test your backup it wouldn’t have been an issue.

Other than continuously copying files to new media every so often, are there reliable digital mechanisms for long-term storage (say, 50 years)?

M-Discs have solved it. Obviously, we don't have 2000 years of data rot to verify it truly works.

How long do the disc readers last?

The reader is an ordinary DVD or Bluray drive. It's safe to assume these will continue to be manufactured for 20 more years and if they have solid state components, a shelf life of 100 years afterwards.

How is it safe to assume that? Who is still manufacturing Betamax tape decks?

You have a point, but considering Betamax tapes only stopped production in 2016, (the recorders in 2002) and the proliferation of dvd / Blu-ray, etc is much higher I think 20 years is safe. 30 however…

But is there any format you can promise every part of it will be in production in 20 years ?

Amazon still lists some refurbished vhs players but…

It doesn't seem very reasonable to draw conclusions about longevity from any comparison of a rather complex machine like a Betamax deck (that cost a huge amount of money to buy) to something rather simple like a modern Blu-Ray drive ($100, ish).

It seems much more reasonable to assume that Blu-Ray hardware will continue to be produced until something else (what?) actually supplants it for cold, off-line data storage.

(Also, too: The compact cassette is dead as fuck and has been for decades, even with the niche resurgence in recent years. Yet new machines are still being produced, and so is new tape stock. The format is 61 years old.)

Disk drives are not solid state. They have moving parts, motors and belts that can fail.

> M-Discs have solved it.

I have an M-Disc writer and some M-Disc DVDs, so I took note of this submission ~2 years ago:

PSA: Verbatim no longer sells real M Discs, now puts regular BD-Rs in M Disc packaging - https://old.reddit.com/r/DataHoarder/comments/yu4j1u/psa_ver... / https://news.ycombinator.com/item?id=33593967

Weren't BD-R M-Discs never really M-Discs? The retention only applied to DVDs?

Object storage in the cloud is likely to succeed there, but then cost and security issues arise.

If data are encrypted, then managing keys is another pain/cost dimension.

At the several decade point, keeping copies at multiple vendors becomes a discussion point, since even Google and Amazon are not likely to be immortal, and that Ukrainian data center might experience physical security challenges.

There was days where a complete bunch of cloud users lost all their data. So, no, this will not succeed too.

No.

There are multiple problems. Storing stems or a rendered mix can be fully durable with some care to replicate it sufficiently. But what of for instance the session files, which are likely tied to custom hardware which eventually becomes scarce.

Object storage is a fine proposition for long term retention but it does nothing for the organizational problem that someone needs to continuously pay the bill and ensure the provider didn't lose anything, and that can easily get lost in M&A, estate liquidation, etc.

The bottom line is, if something is worth saving, you need someone to take on the role of archivist that will balance the technical and economic changes that go with preservation. There is nothing passive about it unless hope is the strategy.

DNA Data storage looks promising [0].

“Another challenge in conventional storage media is their unsuitability for long-term storage, with optical discs, solid-state drives, and hard-disk drives having lifespans of 25 years, 12 years, and 10 years, respectively […] Moreover, the stability of DNA was proved by the successful recovery of ancient DNA under burial conditions. The studies have shown that preservation of DNA does not require additional energy for data storage.“ [1]

[0] https://www.nature.com/articles/s41598-024-58386-z

[1] https://link.springer.com/article/10.1007/s13534-024-00386-z

Oh interesting. Let me store this DNA sequence that is occasinal the sequence of the spanish flue...

No, DNA is terrible. Microsoft's Project Silica seems to be the contender for indefinite, maintenance free, storage. But, there's the whole "It's Glass" issue.

Release the files to the fans?

Want longer? Seems like MS has a solution?

https://youtu.be/-rfEYd4NGQg?si=QoCve6CAPajmmBiX

There are high density binary microfilm optical formats on archival grade film stock that should be stable for several hundred years. Although tbh I'm an M-DISC guy.

https://en.wikipedia.org/wiki/Optical_tape -- https://en.wikipedia.org/wiki/Write_Once_Read_Forever

Microsoft is working on storing in glass tiles, which should last up to 10.000 years [1]. Its not commercial yet though.

[1] https://unlocked.microsoft.com/sealed-in-glass/

Yes, you can chuck your SSD into a freezer. Data retention time increases exponentially in lower temperatures, so keeping it in a regular +4C fridge is enough to extend retention by decades.

Just remember to heat up the disk before writing and after storage.

https://www.ni.com/en/support/documentation/supplemental/18/...

What about frost?

If my three year old 350€ fridge has a no frost option that never failed or had hiccups in all that time, I assume the industrial one bought to store drives in that hypothetical situation would do to.

You can generally drench electronics as long as they aren't electrically loaded

Corrosion might be a concern but meh, wet for a moment doesn't matter. The drive forgets that part

If truly concerned, treat it like a slab of meat and put it in a bag

>If truly concerned, treat it like a slab of meat and put it in a bag

...of rice.

This is shaping up to be a delicious savings plan

SSD: 4/10

With Rice: 6/10

Thank you for your suggestion

This is why copyright can't be this long.

You're prohibited from duplicating physical media well past the point where it is likely to have degraded.

Thus effectively for much data the copyright balance no longer exists. Much work that should enter the public domain is instead merely wasted. Promotion of the arts and sciences is no longer served.

Spins me out that a storage and archive company didn't think to make regular copies (if I read the article correctly and they, rather than the client, were at fault)

Reply on news.ycombinator.com