Newly recovered Ground Zero photos show why you should back up your CD-Rs now | Hacker News Reader

Newly recovered Ground Zero photos show why you should back up your CD-Rs now

theverge.com

69 pointsTomte61 comments

Direct link to the source: https://twitter.com/textfiles/status/1138537238537625601

Direct link to the images: https://www.flickr.com/photos/textfiles/albums/7215770899728...

And what a title! The photos do not show the need, but the need is shown by some of the CD-Rs on which the photos are stored being faulty. Ground Zero refers to 9/11 here, not the nuclear bombing of Japan in WW2.

My most popular blog post is a test I made, reading 400 CDR:s of backed up data. Most interesting is people discussing their own strategies in the lengthy comments.

https://www.rlvision.com/blog/how-long-do-writable-cddvd-las...

Anecdotal evidence, but I've recently gone through my ancient CD collection (in so doing, I found Ubuntu 6.04 installers, AOL dial-up CDs and other relics) and archived them with GNU DDrescue. It had no problem reading all bar a handful of CDs from the mid-to-late 90s and early 00's - the failure rate was much lower than I expected. The failures were mostly completely visually identifiable too - the layers had started to separate.

I've also been backing up personal photos to DVD annually for quite a few years. Each disc contains encrypted Duplicity files, and a PAR2 archive to assist with any deep failures of the on-medium parity. I also usually include a disk or two of pure parity archives, to handle a total failure of any disk or two in the middle.

Each year I check the price of DVD vs. Blu-ray and when factoring in the cost of the burner DVD seems to stay just on top for $/GB. It doesn't help that I've still got spare DVDs lying around, and people basically give away the blanks these days.

This is not my only backup strategy, but it is the only one that is cold, read-only, and offline.

> Each year I check the price of DVD vs. Blu-ray and when factoring in the cost of the burner DVD seems to stay just on top for $/GB. It doesn't help that I've still got spare DVDs lying around, and people basically give away the blanks these days.

Cost is one thing, but does the capacity not bother you too? DVDs are so small in comparison to BDs. Not to mention the time you have to waste switching discs (or reading/writing them, if relevant).

Burning tens of DVDs is a little tiresome, but it is well enough scripted that it is easy to slowly swap them in and out while focusing on another task - the brief interruption is barely noticed.

It also means that the failure of a whole disk has a lesser impact on the whole.

Edit: that said, I do have a spindle of 100 CD-R that were given to me. I haven't had the motivation to use them up yet.

There is also https://en.wikipedia.org/wiki/Dvdisaster which can automatically create par2 parity when you write DVDs or Bluray. The other important thing to do is always write and verify, a lot of disc errors are there from day 1 and not from the passage of time.

> It had no problem reading all bar a handful of CDs from the mid-to-late 90s and early 00's - the failure rate was much lower than I expected.The failures were mostly completely visually identifiable too - the layers had started to separate.

It was mostly cheap CD-R from that period, and frankly you needn't wait 20 years for them to fall apart, they'd start failing under 5. Quality CD-R from good brands survived just fine.

I have a CD-R of "alternatively obtained software" from December 1995, which I keep around mostly out of curiosity. It's got some forgettable old games and MIDI sequencing software on it, nothing interesting at all.

But I'm wondering when it'll actually start deteriorating and ultimately fail. So far it seems to still be OK.

I would honestly just copy it as an iso with something like dd.

There's nothing of any particular value on it. I'm keeping it around as an experiment, to see how long it'll last before the data is unreadable.

CDs? or CD-Rs?

Your Ubuntu 6.04 installer is probably burnt onto a CDR, unless you bought a pressed copy.

The AOL dial-up CDs are most certainly pressed CDs, and would have a far longer life.

I've seen bit rot in as little as 5 years on cheap CD-Rs.

It was a mix of both - the Ubuntu 6.04 was in fact one of the freely distributed pressed ones, but many were home burned CD-Rs - photos, software, data backups, etc.

I wonder about this: how do people archive their information, especially photo and video data. I use an array of mirrored HDDs in different locations with everything on it, plus parts of it on a cloud hosting service. It isn't very convenient though and it seems there should be a better solution. How do people in here do it? Most solutions (for me) fall short on quick retrieval and possibility to visually browse photos.

I buy new hard drives every year and copy all the data to them.

Edit: It's not just the longevity of the drive (though after 20 years they often don't work at all). Drive interfaces change over time, and I have drives where there is no way to hook them up to a newer computer, and my old computers failed due to capacitor problems.

For example, good luck trying to read a 5.25" floppy. You'll need to build your own hardware and write your own device driver to do it.

Just keep copying it all forward every year. Multi-terabyte hard drives are cheap.

The philosophy of just copying data makes sense, but your justification not so much.

> Drive interfaces change over time, and I have drives where there is no way to hook them up to a newer computer

20+ year old PC drives will have IDE interfaces, which you can of course get a USB adaptor for very cheaply and I have several laying about. Really old PCs or non-PC hardware (e.g. the Amstrad CPC 6128 in the 1980s had an after market hard disk option) will need a different adaptor, but it's still just not that hard, and certainly shouldn't be categorised as "there is no way".

> For example, good luck trying to read a 5.25" floppy. You'll need to build your own hardware and write your own device driver to do it.

You can easily buy refurbished 5.25" drives, and you can convert the data signal (which for a PC was basically the same as for 3.5" drives, but with a different physical connector) to USB. They're bulky and they need a lot of power at 12 volts, which is probably why off-the-shelf 5.25" USB floppy drives aren't a thing unlike for 3.5", but it won't require that you "build your own hardware" in any real sense.

You're only going to need to go writing device drivers if you've got some crazy disk format, and it'd have to be pretty crazy because the Catweasel already covered a lot of that territory (a Catweasel was a thing for Amigans, whose disks are a crazy format, to let them use PC disk drives to read their disks years ago, the last ones have an FPGA in them so that the uploadable firmware can decide how to control the drive).

I have several 5.25 drives laying around. The connectors still fit on them to the motherboard, but the BIOS no longer recognizes them, though the BIOS will recognize 3.5 floppy drives.

I can't find any device that connects the drive with the computer that doesn't require writing a device driver.

It's not a crazy disk format, it's DOS format.

As for hard disk drive interfaces, I have a couple off the shelf USB interfaces to IDE drives. The connectors fit, but they don't recognize the older drives.

I can't speak to what's up with the BIOS versus 5.25" drives, beyond to say that I'm confident somebody out there has this working without writing device drivers.

For the USB to IDE thing, my guess would be that the USB interfaces you tried want to do some newer incarnation of ATA than the drive supports and so they can't communicate, at the extreme that could be LBA48 (48-bit logical block addressing, which is from this century and so obviously isn't going to work on say a 20 year old hard disk) but it could even be DMA, which was optional in early ATA specifications and might have been skipped by cheap manufacturers back in the day.

I don't think logical addressing itself was ever optional, but I guess it's possible that a _really_ old IDE drive might not implement it and yet work anyway in DOS which didn't originally use LBA mode. Doing CHS mode really would be asking a lot, compared to PIO (the alternative to DMA) which really a competent implementation of that USB adaptor ought to be able to do, slowly like a real PC.

> I'm confident somebody out there has this working without writing device drivers.

I google search for it now and then, and come up with nothing that can be simply installed and used.

In the early 80's, I had a PDP-11 with an 8" floppy drive. When I bought an IBM PC, I used Kermit to transfer my files to the PC over the serial cable. Fast forward 30 years, and I had put my PDP-11 Empire game up on github. Someone emailed me that it was missing a file. I was dead in the water, having gotten rid of the 11 long ago.

I recalled my roommate in college might have one. Called him up, he said he was about to discard it, I called him just in time! He said it was unlikely to work (been a long time since it was powered up), and unlikely to be able to read those ancient floppies, but was willing to try.

I had kept the floppies in the attic, and mailed them to him.

They all read perfectly! He sent me all the files on them, and bit images of the disks.

Thank you Shal, and thanks DEC for making machines and floppies that still worked.

https://github.com/DigitalMars/Empire-for-PDP-11

I wish I'd kept my 11.

> For example, good luck trying to read a 5.25" floppy. You'll need to build your own hardware and write your own device driver to do it.

It's really not that hard, assuming some older PC format. Motherboards up to the 486DX4/120 generation had onboard controllers for the cable to interface to a standard AT type, half height 5.25" drive. I had one in my PC at the time. They read 360KB disks fine too.

From that sort of computer, which can run DOS 6.22, it's not too hard to use it to transfer the data to a modern machine.

https://rover.ebay.com/rover/0/0/0?mpre=https%3A%2F%2Fwww.eb...

If you mean good luck trying to read a 5.25" disk due to magnetic degradation on the media,that's another problem.

> From that sort of computer, which can run DOS 6.22, it's not too hard to use it to transfer the data to a modern machine.

Well, yes, if you have an old computer with 3.5 and 5.25 drives, you can transfer to a 3.5 and read that on a modern computer. The trouble is, my old computers all failed due to capacitor problems.

I can buy a 3.5 floppy usb drive, plug it into any computer, and it works. Not so for 5.25.

> If you mean good luck trying to read a 5.25" disk due to magnetic degradation on the media,that's another problem.

I haven't had much issue with that. About 20 years ago, I transferred my thousand or so floppies one by one to CD ROMs, and later to hard disks. Only a couple were unreadable. I've since found a handful of others, but no way to read them any more.

So this isnt a way to actively use a 5.25" floppy for storage... but if you have data on one you want to save this is an option:

http://www.cbmstuff.com/proddetail.php?prod=SCP

Thanks for the tip! But it doesn't seem to present the floppy data as a file system :-(

Processed photos and videos: Google Photos and YouTube. As an insider I know that I can't afford to match the availability of those and having the live display copy indestructible is soooo convenient. Downside: I don't think YouTube will give you back verbatim input material. Photos will, but it can be expensive.

Raw media material after processing: single local HDD and a remote raid.

Home directory: two local time machine HDDs and a remote raid copy of head.

Unpublished code: home directory + a private git server.

Published code: as unpublished + GitHub (which I assume to be indestructible too).

I agree that it's hard to beat the availability of Google consumer services... But that is not backup.

A backup serves as a fallback if for any reason the primary source has become unavailable.

In Google's case such an even can and does occur when your account gets banned because of some random event which you had hardly anything to do with or because your user was compromised.

A backup let's you recover from that and if you don't have that, I hope for your sake you don't mind losing the data too much, because the previous situations happen quite frequently if you actually listen on Twitter, Reddit or whatever social network you frequent.

As I wrote: Google services are the display copy. Conveniently it's overwhelmingly likely to survive for years upon years without any effort. But it can disappear, for reasons you mentioned, in which case I can recover the input media from the options in the second paragraph.

Google Photos is most decidedly not a viable backup for photos and videos.

Google Takeout is broken or flakey, and had been for at least a year. Good luck downloading any archive before it falls with "transfer failed: bad authentication" 30 minutes into the transfer. Of 30 tarballs, only one actually completed successfully. And the images in the tarballs aren't the same bits that I uploaded (via my pixel, so it's supposed to be "original quality," not "high quality").

Pulling your files via the API results in corrupted metadata. Time and GPS location tags are stripped.

YouTube videos are, by design, reencoded with very low bitrates.

I store my important documents, music collection etc. locally on my PC, regularly synced to a NAS, automatically synced to pCloud, and every week I sync to an external hard drive that I keep in my locker at work.

Maybe it's overkill, but it really isn't much work to bring the disk home, fire off an rsync script and bring it to work again the next morning. And external hard drives are ridiculously cheap.

For photos, my phone uploads them to Google Photos automatically. I've been considering moving that to pCloud as well.

Once a year, before christmas, you chose about a hundred of the best photos, print them in a beautiful album, send copies to your grandmothers, and delete the rest.

I love this approach.

It features (with some overlaps):

1) Less is more.

2) Data hygiene.

3) Present for grandfather/grandmother/family.

The problem is that the more photos you made, the more difficult it becomes to pick a hundred. We leave behind so much data trails every day, including data which is no longer relevant. We constantly have to decide which data we keep or 'consume'. The problem is only becoming worse throughout the years (in history). Data is more easily available, and more easily able to be made (or fabricated!) now than 50 years ago.

I've had a policy of keeping one photo a day and discarding the rest for some time now. It's quite freeing to not have to worry about keeping a huge digital archive.

In times where storage space is cheap is there really a difference between keeping one picture safe or keeping 100 safe? If you can keep one safe, you can keep 1000s safe with the same kind of process.

I think there is, my wife's photo archive has 30,000 images. Too big to zip up and put in an email, too big to put on a cheap USB stick, too big to quickly upload to S3, too big to properly catalogue and index, too big to really appreciate.

Less is more.

But how can you keep even a single picture safe? Are you able to read today any of your files from 20 years ago? Not in a theoretical sense.

I was mostly referring to the previous comment about only keeping one picture and not a huge archive.

But I don’t see a problem opening images from 20 years ago. If you are worried just keep a jpeg or raw viewer app in the archive too. Worst case there will probably be emulators to run an ancient OS where your image viewer works and you can convert your images.

I have a small laptop running a few services for home use, incl. a DLNA server and network storage. Media dropped on a specific network drive will be available over DLNA.

Makes it easy to look at photos from computers and DLNA capable TV's, but it doesn't work for archival purposes as the laptop used as a server isn't really redundant.

Our own laptops are backed up using a cloud backup solution.

It would be neat to have a service that converts digital photos to the equivalent of film or microfilm, an analog medium that can be converted back to full size pictures in a hundred years. It would have to be scalable and cheap though, given how easy it is to take pictures nowadays.

But yeah, spend $100 / year and get a big film roll of your photo archive, that would be neat.

A few of my co-workers have cross-site backups arranged with siblings or parents. Each house runs a few local servers and mirrors the backups to the other house.

This relies on having a trusted party outside of your natural disasters zone, but it's pretty cost effective if both parties wanted to run home servers anyway (e.g. Plex, homelab)

A huge shame. I backup my optical media to S3, but I also use M-DISC "mineral discs" which hopefully will last much longer. Sadly they were not available back then.

Do you mind sharing your monthly S3 bill and it’s rate of change?

It's around 10 dollars, but then I have less than 1 TB. I recently starting backing up my RAW files remotely, so this number is set to increase significantly (!) I am very interested in S3 alternatives. I haven't looked at the Amazon Glacier options either, but I suspect the retrieval times make it awkward to query/verify backups.

Thank you. My pleasure and problem is my spouse is a professional photographer ;)

I can recommend wasabi.com, they are a lot less expensive but also offer a full S3 api.

But do you want to trust a company you never heard of (in my case) over one that every big company relies on with your backups to safe a couple of bucks / month?

Yes. People get burned every day by well known big companies.

Side note: I’ve heard about Wasabi for a long time; they actively advertise on APMs Marketplace audio program about how much cheaper they are then S3 for storage and transfer. Backblaze B2 would’ve also been an acceptable alternative to S3.

> Yes. People get burned every day by well known big companies.

Usually from free or niche products of well known big companies. S3 will probably not be deprecated over night without a appropriate notice period as half the internet depends on it in some way as you can see on S3 outages.

I used to use S3 but quickly realized how expensive it is and switched to Hetzner. The more data you need to store and transfer, the more pronounced the difference is.

Long-term archival / storage is not the use case for S3, that'd be more Glacier or B2's thing.

The storage itself is competitive with Hetzner's storage solutions (nextcloud or storage box), though you'd pay for traffic on top of that.

Glacier is wonderful. I am able to store basically everything I want and it costs me less than $1/mo

The cost in glacier is if you ever need to retrieve data

Right which is free for the first 10GB each month and then is $0.01 per GB after that for standard retrieval.

What's Hetzner's redundancy like?

I rent a 40 TB box with Hetzner set up as a 2x20TB RAID1 paying €78. I can be sure I won't pay more than that each month. With S3, I can be sure I'd pay at least $560 each month for 20 TB plus the traffic - and this made frequent synchronization pretty unattractive to me. So far their service was very reliable, with hard drives failing a couple of years, but they were replacing them very quickly (half an hour or so) so for me the availability of Hetzner is quite close to what I had with S3 - but I sleep better at night.

Archival grade blu-ray or mdisc, those should last surprisingly long. Even normal blu-ray is ok, but the very cheap ones are not, those use organic dye. The rest is usually ~5-8 years.

I use MDISC media for photos. I wonder what everyone's experiences are, especially for the 25GB and 100GB Blu-Ray variants - have you seen them fail? It'd be great if MDISC are indeed as great as they claim to be and I don't have to worry about copying the stuff over to some other media for 50 years or so.

PS: Yes, I have an external optical drive with USB 3.0 (and a USB A and USB C cable) to go with the media to be able to read them later

My USB Blu-ray drive failed after two years and new ones cost about four times more than they used to. I shrugged it off and added one more hard drive of redundancy instead.

While hard drive lasts, the trouble of disk based storage lies in the filesystem. I'd recommend running ZFS for long term storage.

When do you predict Windows will no longer read FAT32? My first real computer was Windows 95b and that formatted FAT32 disks, and even my current 7 installation reads and writes FAT32 on flash drives

I was referring to bit flips and data corruption. Mirrored zfs can correct these, the rest of the filesystems can't.

Flash drives are terrible mediums for backups unless you power them every month. When they loae charge they lose bits permanently.

Do flash drives every "refresh" unmodified storage? I'm not super familiar with the behavior of cheap flash storage controllers

I just keep all my archives in zfs mirror disks, which I every 4 years buy 2 new of and copy them to there. The old ones are not deleted, just stored offline.

Then I also have usb-memory sticks which I gave to family.

Reply on news.ycombinator.com