Hacker News Reader

Comment by LaGrange | Hacker News Reader

> There are no (stable) alternatives. BTRFS certainly not, as it's "under heavy development"¹ (since... forever).

Note that they don't mean "it's unstable," just "there are significant improvements between versions." Most importantly:

> The filesystem disk format is stable; this means it is not expected to change unless there are very strong reasons to do so. If there is a format change, filesystems which implement the previous disk format will continue to be mountable and usable by newer kernels.

...and only _new features_ are expected to stabilise:

> As with all software, newly added features may need a few releases to stabilize.

So overall, at least as far as their own claims go, this is not "heavy development" as in "don't use."

View on HN

Some features such as Raid5 were still firmly in "don't use if you value your data" territory last I looked. So it is important to be informed as to what can be used and what might be more dangerous with btrfs

Keep in mind that RAID5 isn’t feasible with multi-TB disks (the probability of failed blocks when rebuilding the array is far too high). That said, RAID6 also suffers the same write-hole problem with Btrfs. Personally I choose RAIDZ2 instead.

To be sure, it's entirely feasible, just not prudent with today's typical disk capacities.

Feasible is different than possible, and carries a strong connotation of being suitable/able to be done successfully. Many things are possible, many of those things are not feasible.

> Keep in mind that RAID5 isn’t feasible with multi-TB disks (the probability of failed blocks when rebuilding the array is far too high).

What makes you say that? I've seen plenty of people make this claim based on URE rates, but I've also not seen any evidence that it is a real problem for a 3-4 drive setup. Modern drives are specced at 1 URE per 10^15 bits read (or better), so less than 1 URE in 125 TB read. Even if a rebuild did fail, you could just start over from a backup. Sure, if the array is mission critical and you have the money, use something with more redundancy, but I don't think RAID5 is infeasible in general.

Last time I checked (a few years ago I must say), a 10^15 URE was only for enteprise-grade drives and not for consumer-level, where most drives have a 10^14 URE. Which means your build is almost guaranteed to fail on a large-ish raid setup. So yeah, RAID is still feasible with multi-TB disks if you have the money to buy disks with the appropriate reliability. For the common folk, raid is effectively dead with today's disk sizes.

Theoretically, if you have a good RAID5, without serious wire-hole and similar issues, then it is strictly better than no RAID and worse than RAID5 and RAID1.

* All localized error are correctable, unless they overlap on different disks, or result in drive ejection. This precisely fixes UREs of non-raid drives.

* If a complete drive fails, then you have a chance of losing some data from the UREs / localized errors. This is approximately the same as if you used no RAID.

As for URE incidence rate - people use multi-TB drives without RAID, yet data loss does not seem prevalent. I'd say it depends .. a lot.

If you use a crappy RAID5, that ejects a drive on a drive partial/transient/read failure, then yes, it's bad, even worse than no RAID.

That being said, I have no idea whether a good RAID5 implementation is available, one that is well interfaced or integrated into filesystem.

I have a couple of Seagate IronWolf drives that are rated at 1 URE per 10^15 bits read and, sure, depending on the capacity you want (basically 8 TB and smaller desktop drives are super cheap), they do cost up to 40% more than their Barracuda cousins, but we're still well within the realm of cheap SATA storage.

Manufacturer-specified UBE rates are extremely conservative. If UBE were a thing then you'd notice transient errors during ZFS scrubs, which are effectively a "rebuild" that doesn't rebuild anything.

Btrfs has many more problems than dataloss with RAID5.

It has terrible performance problems under many typical usage scenarios. This is a direct consequence in the choice of core on-disc data structures. There's no workaround without a complete redesign.

It can become unbalanced and cease functioning entirely. Some workloads can trigger this in a matter of hours. Unheard of for any other filesystem.

It suffers from critical dataloss bugs in setups other than RAID5. They have solved a number of these, but when reliability is its key selling point many of us have concerns that there is still a high chance that many still exist, particularly in poorly-exercised codepaths which are run in rare circumstances such as when critical faults occur.

And that's only getting started...

There's differing opinions of BTRFS's suitability in production - it's the default filesystem of SUSE on one hand, on the other RedHat has deprecated BTRFS support because they see it as not being production ready and they don't see it being production ready in the near future. They also feel that the more legacy linux filesystems have added features to compete.

Check what features of BTRFS SUSE actually uses and considers supported/supportable.

It's also the default file system of millions of Synology NASes running in consumer hands (although Synology shimmed on their own RAID5/6 support)

Kroger (and their subsidiaries like QFC, Fred Meyer, Fry's Marketplace, etc), Walmart, Safeway (and Albertsons/Randalls) all use Suse with BTRFS for their point of sale systems.

Synology uses standard linux md (for btrfs too). Even SHR (Synology Hybrid RAID) is just different partitions on the drive allocated to different volumes, so you can use mixed-capacity drives effectively.

Right, instead of BTRFS RAID5/6, they use Linux md raid, but I believe they have custom patches to BTRFS to "punch through" information from md, so that when BTRFS has a checksum mismatch it can use the md raid mirror disk for repair.

Facebook runs on btrfs: https://facebookmicrosites.github.io/btrfs/docs/btrfs-facebo...

But then, your personal requirements/use cases might not be the same as Facebook's. (And this does not only apply to Btrfs[1]/ZFS, it also applies to GlusterFS, use of specific hardware, ...)

[1] which I used for nearly two years on a small desktop machine on a daily basis; ended up with (minor?) errors on the file system that could not be repaired and decided to switch to ZFS. No regrets, nor similar errors since.

Reply on news.ycombinator.com