Hacker News Reader

It would be neat if filesystems like btrfs or ZFS could expose their internal checksum of the file data to userspace, to quickly see if two files are having identical content (eg. think of rsync). (Assuming the hashes computed internally the filesystem are actually purely based on the file data, and not for example metadata like block pointers. But, dedup-capable filesystems should surely have such checksum internally...).

View on HN

The problem with that is for good performance reasons that checksum is typically on smaller blocks than the whole file. For example ZFS splits files into a maximum of “recordsize” (128K) by default and checksums each block.

Not so likely whatever remote target you are looking at has the same blocks and checksum algorithm.

On the upside you can often ask such file systems to take two snapshots and what changed between them. Or to export some kind of differential to transform the original snapshot to the newer snapshot. Both ZFS and Btrfs can do those.

For ZFS, we have the pool-wide transaction number, which increments with every disk write. Couldn't you use that as the "time" for the last modification to each file so long as you have a starting transaction number to compare it against?

Reply on news.ycombinator.com