Hacker News Reader

Comment by skrause | Hacker News Reader

No checksumming?

I have a theory on this regarding consumer devices:

Checksumming drives a lot of support calls that would otherwise not happen. As long as the errors are in media files and not meta-data, most consumers are going to be oblivious to bit rot in their downloaded movies and photos.

Enabling checksumming is going to reveal a lot of errors that would otherwise be silently ignored (spoken from experience running a ZFS media server)

They could just checksum metadata, which is cheaper to store redundantly.

ReFS already does this when you're running it on non-redundant storage - integrity streams are only enabled on mirrored/parity configurations.

Just because you do checksumming doesn't mean you have to report those errors in a way that might frighten a non-technical user. How many regular Mac users sit there looking at the syslog all day?

That is insane! I just so strongly assumed that any new advanced filesystem would have checksumming, I read the entire document without realizing it is never mentioned.

It would be unbelievable if they really didn't have checksumming. Could it just be that they haven't documented it yet? That seems weird and unlikely, but... not was weird and unlikely as APFS not having checksumming.

It may, or may not, have it but I very much doubt there isn't room to retrofit it where appropriate.

I also didn't see any mention of compression. Hopefully those are at least being worked on and just didn't make it into the docs yet.

HFS+ already supports compression and it says APFS would support nearly all of the features of HFS+, so there's hope.

I had forgotten about that. I don't really know how HFS+ compression works, so I'll wait and see APFS turns out.

HFS+ uses zlib (gzip).

As far as I know only for read-only volumes...

Checksumming is the thing that I want the most from a new Apple FS. I really hope they include it at some point.

Heck, if someone's going whole-hog on file integrity, I'd really like them to support the creation of files that are stored "expanded" by a fountain-code.

Rather than having to RAID-mirror every block on my disk, I'd like to be able to pick just some files and say "please store those ones slightly more redundantly, such that they're protected from bit-level disk errors—they're important."

something like zfs copies=n? I used this on a drive with tons of bad sectors and it worked flawless - ZFS even spreads the writes throughout the disk. However AFAIK this is a property of using merkle hash trees (just add multiple leaves) and it's probably unlikely that Apple went this way.

Can you implement COW filesystems without some kind of checksumming? Or have I been in ZFS-land for too long?

Yup. Nothing intrinsic to it that needs checksums.

    zfs set checksum=off rpool

Sure, just assume that there's no such thing as bit rot.

I think you're mixing up checksumming and content-addressing.

Yes, what's more difficult is implementing data checksumming without CoW. What you may not be used to in ZFS is a nodatacow option (per file or per filesystem) which does exist on Btrfs and it implies nodatasum (no checksums for data, metadata is still always checksummed and cow). Conversely, nodatasum does not imply nodatacow.

Checksumming makes no sense on modern hardware. Hard Drives and SSDs use CRCs already.

When a sector goes bad on a hard drive, the firmware will mess about retrying and altering the analog amplifiers to try to get the signal back. If it gets the data back, it might "recover" by moving the data to spare sectors. Without a CRC, the firmware would have no way of differentiating between a sector read correctly and one that is read erroneously.

Checksumming makes sense when the checksum accompanies a pointer to another block. i.e. a directory entry says subdirectory contents are found in block 1027 /with checksum 0xdeadbeef/.

If block 1027 is errantly overwritten due to a bug in the filesystem, the block driver, the DMA subsystem, or the device firmware, the FS will know as soon as it goes to fetch block 1027 that something went wrong. This is true even if the block is still internally consistent; perhaps the write was misdirected, another sector was incorrectly read, or a failure in the flash translation layer caused a newer write to be lost.

If there's any redundancy to the system, whether storing the metadata in multiple places on the disk, or on different disks in an array, the FS can then check all the others and, crucially, detect which version is correct.

Well it happens enough in practice, so it must come from somewhere. Maybe its the shitty SATA cable, or RAM bitflips, or the disk.

I'll gladly spend 1% or whatever space overhead to notice it.

SSDs use CRC to deliver their probabilistic data storage approximation. FS level checksums are, among others, for when that fails.

FS level checksums also cover bitflips on your bus during the write phase (the drive will crc the already bad data) or during the read phase (you receive different data than your drive sent).

At least with bigger SATA disks I see regularly checksum errors on btrfs/ZFS and a rising bad sector count. The disk usually replaces the sectors on the next write but when reading it returns either zeros or wrong data. It's nice to know what files are affected from bad sectors. It's not common but common enough to have it.

What sort of "bigger SATA disks" are you using, or how are you treating them?

If I see one bad sector develop on a disk for no obvious reason, it's getting replaced ASAP because it's a sign that more will follow very soon.

several hundreds of 4TB HGST enterprise in a cluster at university. Disks get replaced ASAP but I'm seeing the checksum errors often in the logfiles when a disk starts to collapse.

Btw: So far the Backblaze reliability numbers check out (<1% annual failure rate)

Reply on news.ycombinator.com