ZFS won't save you from bit rot
jodybruchon.com
jodybruchon.com
A big part that's missing is that errors might not come from the actual disks but from the connectors, cables or IO interface chips (HBA card for example). ZFS will detect this, and replacing the broken hardware and a scrub and you're back in business. At worst you might lose a few files if the data was written with the bad hardware. I've experienced this several times in my 10 years of using ZFS, and I've lost two files total (due to intentional lack of redundancy), the rest was just fine.
There's also no mention of the copies parameter, which while not perfect, does allow for some level of redundancy even on a single drive. ZFS uses 3 copies for internal data structures regardless.
> If your drive’s on-board controller hardware, your data cable, your power supply, your chipset with your hard drive interface inside, your RAM’s physical slot connection, or any other piece of the hardware chain that goes from the physical platters to the CPU have some sort of problem, your data will be damaged. It should be noted that SATA drive interfaces use IEEE 802.3 CRCs so the transmission from the drive CPU to the host system’s drive controller is protected from transmission errors. Using ECC RAM only helps with errors in the RAM itself, but data can become corrupted while being shuffled around in other circuits and the damaged values stored in ECC RAM will be “correct” as far as the ECC RAM is concerned.
> The magic CRCs I keep making fun of will help with these failures a little more because the hard drive’s ECC no longer protects the data once the data is outside of a CRC/ECC capable intermediate storage location. This is the only remotely likely scenario that I can think of which would make ZFS CRCs beneficial.
Still, this happens quite a bit, in my personal experience and from what I've read on the mailing lists etc, so shouldn't be downplayed. I've replaced at least 7-8 cables after ZFS started complaining (could be reseating would help but I have enough cables). That's a lot of potential errors other filesystems which don't do error detection on read wouldn't discover, given that I almost certainly wouldn't have found out until much later in that case.
I've also read plenty of people who've had ZFS reporting errors which went away after replacing a SAS/SATA backplane.
Then you have Controllers/HBA's/Backplanes either going bad, or more often overheating. Put a fan on your HBA. These errors can show up across multiple disks
Bad/weak/noisy power supplies, and system overclocking that effects the PCIe bus and attached (like messing with BCLK) can cause havoc as well. Sometimes a CPU/system might have an undocumented bug specific to that system that can cause sanity loss trying to figure out WTF is going on.
I've also noticed (and have come across a few others as well) that if you have enough Samsung SATA 860 EVOs, a particular one may end up persistently generating just a few errors every so often. I don't recall if these were read/write errors or a checksum error, but a clear and scrub will resolve these without incident.
Out of 30+ personal HDDs, I've only ever had one that had started generating checksum errors because it was going to shit. Not counting the laptop HDD that started clicking 20 years ago and taught me a very important early lesson on backups.
Also, it has been updated several times. Most recently in 2020.