Comment by toast0 | Hacker News Reader

Comment by toast0 | Hacker News Reader

Parent Full thread

IMHO, there's not much hassle anymore, unless you seek it out. The FreeBSD installer will install to zfs just as well as ufs. This article seems to not take the least hassle path.

Backups using zfs snapshots are pretty nice; you can pretty easily do incremental updates. zfs scrub is great to have. FreeBSD UFS also has snapshots, but doesn't have a mechanism to check data integrity: fsck checks for well formed metadata only. I don't think ext4 has snapshots or data integrity checking, but I haven't looked at it much.

There are articles and people claiming you need ECC to run zfs or that you need an unreasonable amount of memory. ECC is nice to have, but running ZFS without ECC isn't worse than running any other filesystem without ECC; and you only really need a large amount of ram if you run with deduplication enabled, but very few use cases benefit from deduplication, so the better advice is to ensure you don't enable dedup. I wouldn't necessarily run zfs on something with actually small memory like a router, but then those usually have a specialized flash filesystem and limited writes anyway.

> you only really need a large amount of ram if you run with deduplication enabled, but very few use cases benefit from deduplication, so the better advice is to ensure you don't enable dedup

a lot of people parrot this, but you can always just check for yourself. the in-memory size of the dedupe tables scales with total writes to datasets with deduplication enabled, so for lots of usecases it makes sense to enable it for smaller datasets where you know it'll be of use. i use it to deduplicate fediverse media storage for several instances (and have for years) and it doesn't come at a noticeable ram cost.

> i use it to deduplicate fediverse media storage for several instances (and have for years) and it doesn't come at a noticeable ram cost.

Nice usecase. What kind of overhead and what kind of benefits do you see?

The difference is zfs does a lot of work and makes a lot of promises that it proved the data is good at every step of the way while it's being handled, that other filesystems do not do.

So: "I copied the data and didn't really look at it much." and it ended up being corrupt,

is different from: "I promise I proved this is solid with math and logic." and it ended up being corrupt, complete with valid checksum that "proves" it's not corrupt.

A zfs scrub will actually destroy good data thanks to untrustworthy ram.

https://tadeubento.com/2024/aarons-zfs-guide-appendix-why-yo... "So roughly, from what Google was seeing in their datacenters, 5 bit errors in 8 GB of RAM per hour in 8% of their installed RAM."

It's not true to say that "Well all filesystem code has to rely on ram so it's all the same."

ZFS says "once I've committed to disk, if the data changes, I'll let you know".

This works, regardless of if you have ram errors or not.

I will say that the reported error rate of 5 bit errors per 8 GB per hour in 8% of installed RAM seems incredibly high compared to my experience running on a fleet of about one to three thousand machines with 64-768 GB of ECC RAM. Based on that rate, assuming a thousand machines with 64 GB ram each, we should have been seeing about 3000 bit errors per hour; but ECC reports were rare. Most machines went through their 3-5 year life without reporting any correctable errors. Of the small handful of machines that had errors, most of them went from no errors to a concerning amount of errors in a short time and were shut down to have their ram replaced; a few threw uncorrectable errors, most of those threw a second uncorrectable shortly thereafter and had their ram replaced; there was one or two that would do about one correctable error per day and we let those run. There was one, maybe two that were having so many correctable errors that the machine check exceptions caused operational problems that didn't make sense until the hourly ECC report came up with a huge number.

The real tricky one without ECC is that one bit error a day case... that's likely to corrupt data silently, without any other symptoms. If you have a lot of bit errors, chances are the computer will operate poorly; you'll probably end up with some corrupt data, but you'll also have a lot of crashing and hopefully run a memtest and figure it out.

It would appear that your comment is about ECC itself.

The way I explain ECC RAM and file systems is "Since data is present in RAM before it is given to a file system driver to store and after data is retrieved by the file system driver, the data is only as good as what the RAM can assure." ZFS handles everything once the data is in its purview. It provides various features to ensure redundancies and recoverability in case the underlying hardware fails for any reason.

It's ... pretty nearly the same.

> However, if your RAM is not ECC RAM, then you do not have the guarantee that your file is not corrupt when stored to disk. If the file was corrupted in RAM, due to a frozen bit in RAM, then when stored to ZFS, it will get checksummed with this bad bit, as ZFS will assume the data it is receiving is good. As such, you’ll have corrupted data in your ZFS dataset, and it will be checksummed corrupted, with no way to fix the error internally.

This is more or less true, but the same is true of... anything? If a file is corrupted in RAM, literally any filesystem will just save the corrupted data; that's how it works. The only way I can see for ZFS to somehow be worse than anything else is if a scrub moves things around, in which case yes I suppose you technically are exposed to an extra case of the data being in RAM to risk corruption, but any other time you're way ahead with ZFS regardless.

Reply on news.ycombinator.com