Hacker News Reader

Comment by candiddevmike | Hacker News Reader

Maybe I'm just cynical but I think the ship has sailed for BTRFS RAID 5/6. It's now part of the global mindshare that BTRFS RAID 5/6 == data loss, no one wants to be the guinea pig that proves it works.

Better to direct resources towards bcachefs or ZFS IMO.

View on HN

Ideally we'd be working based on actual information rather than global mindshare. There will be guinea pigs to test it. There are already quite a few people running it despite the warnings.

As much as I want to see a wide use of bcachefs, it's still years away. As someone who actually wants to store data - why would you direct resources to bcachefs which is known experimental, rather than btrfs which plainly documented raid5 as not ready and now may decide to change it to ready... if it is? (I'm donating to bcachefs patreon, but commenting from the PoV of someone choosing a solution today)

Is bcachefs years away? My understanding is the critical chain of code is pretty well tested - bcache.

From widespread use, yes. Bcache was already stable at least 7 years ago. Bcachefs it's not even included in upstream kernel yet, much less any distro as an official option. There's only some plan to start submitting it soon™ and it will take months to complete.

Right but "actual information" here is "developers over and over fail to nail what ZFS and MDRAID did well decades ago". There is no reason to trust their next try will be "the one"

Did well in what context? RAID5? There were enough people with motivation to spend time on it rather than other things for mdadm/zfs. It's not like btrfs devs wouldn't know how to implement it if you paid them to work full time on it.

MDRAID had write hole until 4.4 (2015, https://lwn.net/Articles/665299/) and it had been long awaited back then. ZFS to my knowledge deals with that by variable stripe length that has its own problems but yeah it works. Btrfs changes implementation of the stripe update while preserving the on-disk format (i.e. can't do the same as ZFS without introducing incompatible change), intent log/bitmap have been proposed (which would be the MDRAID approach) but that's another incompatible change. So the 'next one' is at the cost of performance but with same compatibility.

I've been rocking raid 5/6 btrfs for years for my personal data, no problems apart from the speed. A scrub literally takes months.

Yeah same here. Directory deletions are excruciatingly slow too.

Months? I thought it would just go through all the data and verify that the stored checksums match. Even if it fetches filedata more than once to ensure the parity bits are ok, how can it take longer than reading the entire disk a couple times?

Well this is just a desktop not a server, it isn't on 24/7.

ZFS will never be mainlined so it’s beyond useless.

ZFS is widely used by many users and organizations. Just because it is not currently included in the Linux kernel does not mean that it is useless. There is ongoing work to integrate ZFS into the Linux kernel, and in the meantime, many users continue to find value in using ZFS on their systems. As an alternative, you could try using FreeBSD, which includes ZFS as part of the kernel. This can make it easier to take advantage of the features and benefits of ZFS without having to install and configure it separately. Additionally, because ZFS is integrated into the kernel, it can take advantage of FreeBSD's advanced security and performance features.

Enlighten me on how a filesystem with an incompatible license is ever going to be mainlined. Having to use an out-of-tree patchset is an immediate no as far as I’m concerned. I also have no intention of switching to an hobbyist operating system, thank you.

You're already using a hobbyist operating system... Your horse isn't as high as you think.

Linux hasn’t really been a hobbyist operating system for years as anyone taking even a casual glance at a list of contributors would know but sure let’s all pretend FreeBSD is somehow relevant or properly maintained. That’s probably in the same universe where ZFS has a chance of being mainlined and isn’t heavily encumbered by Oracle owned patents.

Why people would contribute to this mess is beyond me but everyone is free to do what they want.

> Linux hasn’t really been a hobbyist operating system for years as anyone taking even a casual glance at a list of contributors would know but sure let’s all pretend FreeBSD is somehow relevant or properly maintained.

You haven't glanced at a list of FreeBSD's contributors, have you?

You're pretending that FreeBSD isn't relevant...

You're probably behind a pfsense firewall right now.

You have probably seen someone playing games on a PlayStation 3 or 4.

You've definitely benefitted from the FreeBSD project without even realising.

Have you taken a casual glance at the list of FreeBSD contributors?

I know FreeBSD is used by Sony on its console, by Netflix and by WhatsApp. Everyone knows because they are pretty much the sole serious contributors to it. I don’t think it’s particularly relevant nor do I think it’s particularly good to be honest but we are getting far from the initial discussion about ZFS.

>Everyone knows because they are pretty much the sole serious contributors to it

You haven't even looked at the list have you?

I recently tried out the Ubuntu native ZFS installation.

I was surprised to see that it taints the kernel when I checked the demsg.

Why is that surprising? It is a difference license. So is the Nvidia driver.

> There is ongoing work to integrate ZFS into the Linux kernel

Could you paste a link to it, please? As far as I know, all efforts now are around ZoL which is not included for legal reasons.

BTW, ZoL is not anymore. ZoL and OpenZFS merged and it's all OpenZFS, sharing a codebase.

Not mainlined = useless?

Ok.

DKMS is fine, and Ubuntu even ships prebuilt .ko files; it works fine in practice.

I'll dump Linux before I dump ZFS

What's wrong with loading the module yourself? Ubuntu is just a command away to get it running with installing zfsutils-linux.

How's that beyond useless?

Ideally on a ZFS replacement that has a compatible license.

Using Ubuntu is hell of a lot faster than waiting for such a file system.

Ubuntu is embracing an extending Linux through it's Snap repo. Dump it now.

Alternatively just continue ignoring snap until it goes away

I do, by not using Ubuntu.

So is using another file system.

We had tried RAID-1 on btrfs, and the experience was not that great either. Our current systems, when they use btrfs, are on top of md (mdadm) RAID-1 arrays.

Do Meta not develop btrfs for their internal use? I don't think community sentiment is a big factor for them.

Meta and related companies have much better systems available for durability than RAID5. I would be moderately surprised if they commonly use any standard RAID level.

Using traditional RAID (or moderate improvements on it, such as provided by btrfs and zfs) to provide drive-level redundancy seems like a waste of time when you also have to worry about redundancy between servers, racks, datacenters and regions.

All of the arguments for why it's better to have something like zfs handle RAID rather than layering something on top of a traditional hardware RAID controller also work for explaining why you should prefer managing storage more globally rather than layering on top of something like zfs—if you have the resources to develop and maintain a true "full stack" storage solution. Which Facebook/Meta obviously does, when they can do things like publish their own spec documents that SSD vendors design around.

I imagine the "does not boot degraded" was mildly annoying to them, until they figured out the rootfsopts flag and deployed it.

That whole thread above debating not booting degraded really made me lose any faith in btrfs.

AFAIK they use it internally, there are articles on lwn.net how, the use cases is for root filesystems and containers. I'm not sure I understand what you mean by the community sentiment, there are examples of code they'd developed internally first and sent it upstream, but in all cases I remember there were no problems. What can happen in the community is e.g. how the patches are organized or if the changelogs are complete. It's of course easier to develop something internally, if it touches other subsystems or if there's enough coverage just for the new code the test/fix/deploy cycle is much flexible. Once it's supposed to go through mailinlists or convincing other maintainers to accept changes it takes longer and must stick to the development cycle. This benefits both sides in the long run.

Their culture of "works for me" attitude makes most stuff they care irrelevant to the rest of the world.

Reply on news.ycombinator.com