Bcachefs File-System Plans to Try Again to Land in Linux 6.6
phoronix.com
phoronix.com
Eg. I could have an SSD and a slow hard disk. Data is written to the SSD first, and eventually mirrored to the hard disk.
If the SSD fails, the hard disk will still be fully consistent up to the point the mirroring process got to. That mirroring would never be allowed to get more than say 5 minutes behind (writes would be throttled if that ever happened).
This lets you have performance and resilience while paying less money. The scheme can be extended with erasure codes to say 32 drives for a huge database so that you could for example sustain any 3 device failures not losing any fsynced bytes of data, or up to 6 failures but you might lose up to 30 seconds of data.
I think many multi-device filesystems could support this mode, but I don't think any do.
Ultimately, the price of SSDs dropped to the level were such solutions filled an ever shrinking niche and they fell out of favor.
However, I don't think either of these systems provide a configurable threshold like "max 5 minutes of dirty cache". Instead, you might have to guess as to how large of a cache device you can risk. The larger the cache, the more volatile data might be lost. But, we generally don't have guaranteed writeback throughput either. So you cannot be sure how long a given cache size will take to fully drain to the second storage layer in the worst case.
When I have used the LVM cache mechanism, I made a RAID1 mirror of two SSDs for the writeback cache layer, and hoped they were different enough devices so that they wouldn't simultaneously fail from the mirrored traffic patterns. In fact, I used a RAID1 of SSDs to writeback cache a RAID1 of HDDs.
But these days, I usually just try to have a large enough SSD to hold my entire data and then use a scheduled backup tool like restic. It copies data to another device but also maintains an archive of multiple snapshots for point-in-time recovery scenarios. I accept up to 24 hours of loss here, using a daily backup procedure.
For really important data where I worry about 24 hours, I might manually replicate after making important file updates. E.g. git push to replicate commits to a remote repo, or rsync to replicate specific subdirs to another machine.
Edit to add: when worrying about 24 hours, it is also worthwhile to think about the probabilities. The extra effort to maintain a low latency backup might end up costing more time than the unlikely potential day of redundant work after a loss. And how long will the recovery take anyway? Do you keep spare parts on hand or expect to spend time procuring hardware and rebuilding your system after a period of inaction due to a broken system...?
The things that I consider too important to wait 24 hours would be non-deterministic things where a recovery method is more difficult than just performing the day's tasks again. E.g. losing recently changed passwords/secrets in an encrypted password store, or losing my notes about other infrequent purchases or bill payments that are in progress. I wouldn't just repeat the steps, but instead have to unravel other recovery procedures for each item.
https://news.ycombinator.com/item?id=32474093
https://discord.com/blog/how-discord-supercharges-network-di...
(This might be able to be done without a specialized file system, although a driver and/or controller might be needed for the mirroring and access.)
I'd be perfectly fine losing 5 minutes of data in case of a disk failure. After all - I can probably recreate it in about 5 minutes.
Anyone who does daily backups is effectively saying "I care about my data, but I'm happy to lose up to a days worth if there is a failure".
Edit: I think you can achieve what you want with current tools, or something pretty close at least, with ZFS. Create a snapshot of your SSD pool every 5 minutes, then stream the delta to an HDD pool in an automated fashion. Delete snapshots more than X hours / days old.
I use the HDD ZFS pool to maintain effectively infinite non-expiring snapshots of my (single device) SSD ZFS pool.
If my SSD fails, I can boot directly from the latest backup on the HDD pool.
I currently run ZFS via Ubuntu on my home devices. I've had a very positive experience with it. My only (minor) complaint is the extra 30 seconds it adds to my desktop's boot time. It's been fantastic for all my other devices, particularly the snapshotting features and ease of which I can ship the snapshots between the devices.
I've read some comments on various forums that seem to hint Ubuntu is moving on from ZFS (see https://discourse.ubuntu.com/t/future-of-zfs-on-ubuntu-deskt... for one example). This is obviously concerning and the lack of support across the whole Linux ecosystem continues to be a nuisance.
Realistically how well could bcachefs serve as a long term ZFS replacement? I know very little about it. I've used btrfs in the past, and between my negative experiences and the repeated anecdotes posted all over the internet to this day I'm hesitant to give it another go.
10 seems a bit short ;)
What are those issues? I've not followed the conversation, but I've been running btrfs for over a decade on multiple machines with different usage patterns without any issues.
I'm genuinely interested. Are those substantiated claims? Or just the general noisy rants you can find about essentially anything people have opinions about?
My understanding is that this is partially resolved now (empty blocks are automatically freed now), but not completely (partially empty blocks are never compacted without a balance, so you can still have a bunch of half-empty data blocks and no room for metadata), so one is a lot less likely to run into similar issues, but it was scarring enough to me that I've never run a BTRFS system again.
I use btrbk to manage my snapshots. It keeps my snapshots pruned to 300, pruning intermediate snapshots, so I can still easily step back 52 weeks, or I can purge them if I want to improve performance and reclaim space. While at the same time, every single snapshot is sent to two compressed encrypted backup images over ssh. The diff algorithm is pretty good, but for some types of files that fragment frequently, it is best to put them on their own subvolumes. My places.sqlite files from my browser testing profiles were adding tens or hundreds of MiB to snapshots, so they went in a seperate subvolume. I don't think the zfs would perform any better.
I don't blame people for having bad experiences with btrfs, it is very inconsistently documented.
I think it is especially annoying how many userland tools keep every single snapshot as a subvolume of the root, so that you have tens of millions of read-only files stored in /.snapshots. Awful.
Has worked well for me so far and saved my butt a few times, and since I don't have a bunch of subvolumes to manage, I'm not overwhelmed. It's nice and simple, and I need nothing other than the default `btrfs` CLI utility.
I suggest moving your filesystem root to its own subvolume (the convention is to call it @root), then optionally mounting the actual filesystem root as like /mnt/fsroot, and mounting it by passing subvol=/. That is where I keep my snapshots. You can still have /home be nested in @root. I have subvolumes for /var/cache, /var/lib/docker, etc. And more fragmented qemu images just go on a NoCOW subvolume with +C set, and I back up normally.
I also highly suggest reflinks. Once you start using them you will use them all the time.
I don't know if zfs supports reflinks yet, at the time it didn't. But combining reflinks with subvolumes solves a lot of problems. Since btrfs doesn't support recursive snapshotting, you can just do cp --reflink=always -rp @root /path/to/new-subvolume, and it will combine the contents of the subvolumes without consuming any more space. And of course, btrfs-send will (usually) not duplicate reflink data.
But I also had around twenty subvolume I needed to snapshot in this manner.
Bcachefs isn’t ready for prime time. I’m reasonably sure it’s better designed than Btrfs, and it’ll get there, but don’t do it yet. If Ubuntu becomes a problem, NixOS has 100% support for ZFS.
I'm planning to do that, but it'll have to be after my vacation; don't want to be running bcachefs on a remote server I can't fix.
For those following along, this sort of response is why I'm optimistic overall. And bcachefs was dramatically higher performance than ZFS, although the lack of an ARC is a particular pain point.
===
It's going to be an interesting bug, I'm pretty sure. It affected two sets of files: The Steam Linux runtime (Solder), and everything in /nix/store, but absolutely nothing else. Actually, I think it's probably a bug in the fsck routine; everything worked perfectly until I rebooted.
Would you care to go into some specifics?
On my middle of the road machines, the limiting factor has always been the drive. Encrypted ZFS has the same transfer speeds as unencrypted ext4, as tested with fio.
My best machine is a zen 3 u laptop, so pcie 3 with a samsung 990 nvme drive. It reads and writes at over 2 GB/s.
When I write data to the setup, bcachefs and ZFS had very different behaviour.
ZFS:
- Writes to the HDDs first.
- Blocks further writes until the first ones are on disk, unless I use sync=disabled. (Which I do; it's safe for my use-case.)
- Is in any case limited by how much can be buffered in RAM.
- Doesn't write to the SSD (which is configured as cache) unless it first reads data into RAM and then that data gets dropped from the in-core ARC.
BcacheFS:
- Writes to the SSD first.
- Writes to the HDDs at its leisure.
- Can be told the NVMe is more reliable than the HDDs.
- Can be configured to leave data uncompressed on the SSD (because even LZ4 is a notable drain at 1.6 GB/s), but use zstd for the HDDs (because it's async so why not, and decompressing zstd at 300 MB/s is no issue).
- Uses LRU instead of ARC, so if your working set is larger than your cache ZFS will have much better behaviour... once it's warmed up.
The bug I hit is apparently fixed now, so I'm likely to switch back to bcachefs more or less ASAP. My working set is not in fact smaller than my SSD... but I'll just get a second SSD, I think.
biggest bugfix in the pipeline is that bfoster just figured out the 'missing backpointer' bug, but that's just in the btree write buffer, it wouldn't cause data corruption - and the fix for that isn't in yet
If it's a bug in fsck that will be fortunate, that'll be easy to track down.
It can provide the same redundancy and safety but with tons more flexibility. For example having different redundancy levels using the same pool of disks and changing redundancy levels on the fly. It does feel like a huge step in in terms of "here are my disks" and then "please store this data this way". bcachefs then just does it in a reasonable and efficient way. Way better than having to set up all of your disk RAID configs upfront.
That being said it definitely still needs stability work. I tried it and it corrupted itself. So I love the idea, the design seems good overall. But it is going to need lots of testing and tweaking to become reliable. However it does seem like a solid addition to the filesystem space and I think being upstream will allow more people to work on it, avoid extra maintenance burden by staying in sync with changes and get the attention it deserves.
I wrote more about why I like the design and how it corrupted for me here: https://kevincox.ca/2023/06/10/bcachefs-attempt/
Wait. What? You'll forgive me, I have no experience with ZFS: why??
I'm not sure if this is the main bottleneck when opening a ZFS pool during boot, but I wouldn't be surprised if it were.
And if it is, personally I'm willing to sacrifice some boot time for some assurances that my data will be safer, but of course, not everyone's priorities are the same as mine :)
(Also, for the record, for me the pool import/opening process is nearly instantaneous on SSDs/NVMe and takes 2.4 seconds on a 12-year-old Intel Atom machine with a single HDD pool).
And you can install it on a lot of distro too: https://openzfs.github.io/openzfs-docs/Getting%20Started/ind...
My current favorite is to install kABI-tracking kmod on RHEL. (But their kABI-tracking is not a guarantee as they uses more.) In fact I move away from Ubuntu knowing that they knowingly ship a ZFS version with file corruption bug, which is introduced by their own patch of OpenZFS.
So if OpenZFS is serving you well, don't worry about the Ubuntu situation because there's plenty better options.
I'm not sure I expect it to end up in 6.6 just because reviews like this do take a long time anyway, but the 6.6 merge window is still a decently long time away from happening. If it doesn't make 6.6 and no major issues are found I could definitely see 6.7 being the one it's in. That'd likely mean it'd be sometime in the late fall early winter (northern hemisphere) which is still great news.
I've been looking forward to bcachefs coming in as an alternative to XFS/ext4 on the lower end of features and as an alternative for ZFS for when you don't need or want all of the stuff it has either. It looks like it should shape up to being a very capable middle ground with a lot of good stuff in it.
Device management is super easy, but is still missing knobs and dials for the most part. Multi device fault tolerance also needs work as I have had to recreate a 16tb array twice now due to journal corruption on power outages.
I'll keep using it and reporting bugs, but I do feel the author is letting some bugs slide to focus on getting the fs in the kernel so that it gets more capable users that can provide patches. I am a sophisticated user, but definitely not a kernel developer.
I'm not involved in any of this besides watching from the sidelines. But it seems to me that Kent's main focus right now is clearly getting the filesystem into the kernel so that it can get the testing and bugfixing it needs. To the point that his determination is annoying all of the veteran kernel maintainers. (Who, to be fair, have a right to be conservative on what they accept into the kernel as they have been burned many times in the past by not being conservative enough.) Even though I am sure he has grand ideas for the future, he seems to be happy enough with the current state of things but hasn't been successful in attracting outside help on his own. Which is why he is trying to drum up interest on the LKML.
I don't think it's likely that Kent is letting known bugs slide, as that would be extremely detrimental toward getting his patches merged.
I started using btrfs back when it was way less stable, but that instability was more than offset by the other features.
The last time I lost a btrfs filesystem was four years ago when it was less stable, but it did not matter because it only amounted to about an hour of logs lost, and I reconstructed it in minutes. So btrfs is a must on any embedded devices. It frees me to use cheaper media, because I can have two copies of all data and metadata, and compression, and fine grained external snapshots.
Zfs is great for making big complex arrays and if you plan out how your are storing your data ahead of time. But it has a steeper learning curve and is more seperate from the underlying OS.
Btrfs fills this perfect niche for me of being a drop in replacement for ext4, and being really flexible and offering some of the nicer features of zfs. (differential snapshots, checksumming, compression)
So I'm excited to try bachefs as an alternative to btrfs.
This isn't a good sign... Journalling should never, if bug-free, lead to data corruption or loss even if there is a power outage.
And, even if there was some bug, one would expect a robust filesystem to say "ah, there is some data corruption here, so we're gonna run an fsck and recover every single file on the disk except perhaps the one or two that the bug clobbered."
This will thwart any filesystem.
SSDs do not reliably report UNC read errors when data can't be retrieved. Garbage or zeros are returned instead.
There's a reason why ext4 and XFS added journal and metadata checksumming. Storage devices just aren't as reliable at informing the kernel when it suspects the data returned is bad.
Recent but no longer current trees, can be partly overwritten when the kernel is informed a super block write was successful. But if the super block write wasn't successful (the device lied), the stale super block on disk points to damaged metadata and recoverability isn't certain.
If it isn't correct, ignore it and move on to the previous superblock. Keep going until you can verify a contiguous 30 seconds of superblocks.
If writes are being delayed by more than 30 seconds, your problems go beyond "out of order".
This does impose the requirement not to overwrite trees that are only a minute or two old. That should not be hard.
Note how much heated debate is happening around how Kent is behaving, rather than the substance of his patchset. The former requires far less effort and knowledge to criticize and attack...
The issue now is there are lots of maintainers who don't like that someone else did something impressive that they couldn't have done, and they are dragging their feet getting the various patches outside `fs/bcachefs` integrated. The design and implementation is basically just Kent, there are others onboarding to help with maintenance. There is a "Wireguard" level of craftsmanship in this project, it's a big deal.
A younger Linus, in another time, might have told everyone to cut the bullshit--if you can't comment on the code, shut up, you're probably incompetent--but that's not the way things work anymore.
It's kinda a shame that there isn't at least some FUSE layer to let you run a filesystem driver from windows/mac/bsd/whatever on linux.
You only need to consider more specialized filesystems if you have unique needs. For example, if you are setting up a file server, you might want a filesystem that is good at disk redundancy and data durability. Or you might need a filesystem that can handle billions of small files. Or you might have a huge database that needs random access to large amounts of flash storage. All of these are very different scenarios that require different (competing) things from the filesystems.
In this case, bcachefs _seems_ to be promising many of the features of ZFS but with a better design and less overhead, potentially bringing these features to general-purpose workloads as well. (ZFS is fairly resource-intensive: in order to make full use of its features, you need a dedicated box for it, or really beef up the hardware on the application server that it will run on.)
For example, the filesystem supports snapshots. This would let you do something like have apt (in Ubuntu) create an automatic snapshot of the filesystem before running an upgrade, so that if the upgrade fails or causes problems you can roll it back trivially.
It also supports native compression, saving users tons of disk space. It supports combining multiple devices together for redundancy, and it can checksum all of your data so that you can know if a file was corrupted without it introducing transparent failures into your system. It supports copy-on-write, so that you can make instantaneous "copies" of files or directories which take no extra space unless you change them.
So yeah, there are a lot of functionalities that we can enable with newer filesystems. ext4 is pretty great all in all, but there are a lot of modern features that modern hardware and concepts enable.
So better compression at the cost of slightly reduced performance for random reads.
Edit: I was paraphrasing the text in the peer linked document:
"As with data checksumming, we compress entire extents, not individual disk blocks - this gives us better compression ratios than other filesystems, at the cost of reduced small random read performance."
In practice being a true extent-based filesystem with compression and snapshotting is one of bcachefs's distinguishing features (snapshotting is rather obvious in a block-based system by comparison). Other systems work similarly at a high level but the details are of course quite different in practice.
How do bcachefs' extents differ from ZFS's logical blocks (whose size are controlled by the dataset's recordsize property) and how do bcachefs' blocks differ from ZFS's physical blocks (whose size granularity are controlled by the ZFS pool's ashift property)?
In contrast, ZFS logical blocks have a fixed size, but may be handled "sparsely" by an underlying number of (smaller) physical blocks. So a 128KiB logical block doesn't mean every 4KiB file takes up 128KiB, naturally. But to change 'recordsize' in ZFS, you also need to rewrite the file, since the logical block size won't be retroactively changed. For good reason, of course. But the indirection between physical/logical makes a big difference in the underlying mechanics.
Snapshots make a lot of sense in a system that uses logical, fixed-sized blocks as the atomic "unit" of data even if there is an underlying physical block layer. For example, COW is trivial -- page is read, written and marked dirty, then written out to a new copy of the block it came from. The original logical block can be left alone, and the cost is understood as proportional to the underlying number of physical blocks rewritten (and some change to update metadata). But how does COW or snapshotting work in an extent system? In a non-COW system, you can update the data inside the extent in-place since you don't copy, but in a COW system you need to copy or potentially split the extent which could be costly. You may actually want to group extents together if they're for multiple versions of a file when doing COW. There is no layer of "indirection" above this, so that's where a lot of the trickiness comes from, I think. If you have indirect "logical pointers" instead, you can defer a lot of work, things like reclamation, fragmentation, splitting and merging, etc.
There are some other interesting design points in the notes[1] I was looking over to refresh myself, but I don't know how some of them relate to ZFS specifically. For example, another thing bcachefs is pretty good at is low tail latency, and the note on eschewing heavyweight transactions/locks is really good example of that. If you want to run 'touch file', you don't need to take a transaction lock, create an inode, then create the dirent, and release the lock. You can instead simply create the inode and then create dirent assuming that they have a proper write barrier between them, and deleting a file is the inverse operation. Then the system can never be left in an inconsistent state, it can only result in a few dangling objects in the key-value store, which can be garbage collected later.
The low tail latency is also assisted by having really small amounts of metadata for each inode, just barely a 64-byte cache line IIRC; so some really weird pathological cases like `ls` in a directory with 500k dirents is way faster in bcachefs than elsewhere. I also wonder if that also falls out from some of its basic design; it basically looks like a POSIX filesystem layered on top of a fast, ordered b-tree, so there's the storage layer and the actual filesystem semantics on top. The filesystem layer probably doesn't actually need to record that much in-band metadata for a given inode when a lot of the underlying consistency and performance issues are handled elsewhere. But that's just me thinking out loud.
Overall, there are lots of differences in the details even if some of the high-level stuff comes across as the same.
A ZFS physical block also records a variable, contiguous range of bytes on-disk, not unlike what you describe with bcachefs extents (although, in ZFS the "contiguous" part can get more complicated due to gang blocks and RAID-Z, I think).
That said, if you don't have compression enabled, then in ZFS the physical size of the block usually does correspond to the logical size, regardless of how much data was written to it (except for files smaller than one logical block size).
Also, if I remember correctly, a ZFS logical block usually only maps to one ZFS physical block, although it can map to up to 3 physical blocks, each of them having the same data (the actual number depends on how many redundant physical copies the logical block is configured to have on top of the ZFS pool redundancy).
But in any case, thank you for your long explanation. Some details still escape me, but I can consult the architecture document you linked if I have more doubts.
So a ZFS block is even larger than a bcachefs "extent".
As an example, in terms of actual on-disk storage, a 128 KB logical block in ZFS can be compressed down to (for example) a 3.5 KB "physical" block and it will only actually consume 3.5 KB of disk space (surprisingly, the block size of a ZFS physical block doesn't have to be a power of 2!).
That said, if you have 4 KB-sector disks then you should probably use a physical block size granularity of 4 KB for your ZFS pool rather than 512 bytes.
So, regardless of the file size, an on-disk ZFS block can actually occupy anywhere from 0 bytes up to 1 MB, with a granularity of up to 512 bytes, depending on how much it is compressed, what the physical block size granularity of the ZFS pool is and what you've selected as the recordsize property (i.e. the logical block size) for the corresponding filesystem in the pool.
According to Allan Jude (OpenZFS & FreeBSD developer), you should never use 512-byte recordsize even on an HDD with 512-byte sectors. Always start from 4 KiBs, probably even from 8.
https://klarasystems.com/articles/tuning-recordsize-in-openz...
I think you meant to refer to the pool's sector size (i.e. "ashift" property) rather than the recordsize.
I agree, but I wouldn't say "never", as that's too strong a word.
It would be useful to use a 512-byte ZFS pool sector size even when the HDD uses 4K sectors if you care a lot about disk space efficiency but don't care at all about performance or HDD wear (for example, if you plan to use a ZFS pool almost exclusively read-only).
That said, I would also recommend a 4K pool sector size in almost all cases, as HDD space is cheap.
I think that post is warning you that it's immutable once done. If you want to be more flexible in the future and can't be sure you won't need to use 4kn/512e disks at some point, yes, use 4kn (or with all SSD storage it gets even messier).
But using 512b can be a significant gain on space efficiency for some people.
I think we're talking past each other. In a practical sense I agree the two filesystems achieve an essentially identical result through very similar methods, as you have said.
I am making a point specifically about the details of the implementation logic of the two filesystems. Bcachefs (as I understand it) combines multiple smaller blocks into a larger extent. ZFS only has one raw record and its corresponding on-disk compressed record, whose size is rounded up to ashift.
The default was to limit it to 1M in case there were weird edge cases with the larger sizes (16M is the largest it'll encode on disk without a format change, IIRC), with larger sizes behind a tunable, but after a few years of people who knew this twiddling it, the default in 2.2 will be that the tunable is set to 16M.
I've said it 1000x -- OpenZFS is already free and open source. Linux kernel developers should just stop punching themselves in face. The ZFS hair shirt is self imposed by semi-religious Linux wackadoos.
[0]: https://lore.kernel.org/lkml/20230626214656.hcp4puionmtoloat...
Bcachefs also has plenty of features which ZFS does not, so I don't see why you're so critical.
Oracle could relicense ZFS at any time, or make an explicit statement about its legal status. They have not done so after 10 years.
Only because I've read this story more than a few times before? See: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
>Oracle could relicense ZFS at any time, or make an explicit statement about its legal status. They have not done so after 10 years.
Any legal difficulty remains from the GPL side, and no suit has been brought in the years upon years of the GPL hair shirts screaming about non-compliance. Unfortunately, the GPL hair shirts don't have much (any) credibility on the subject of what the GPL means.
Bring the suit or STFU. I have no patience for armchair legal opinions from people who won't put their money where their mouth is. Stop spreading legal FUD!
Sorry, if we are looking at the same thing, the earliest time of a HN entry of bcachefs trying to be merged upstream was not even 1 month ago. All the other entries are related to features that are called "merge" and not actual merge requests or the author declaring that the code was up to be merged but not an actual merge request.
1 month ago is nothing in terms of kernel time, so I don't think your argument holds up.
(But that's still a single initial attempt, not "a few" like the OP claimed)
The article also makes it clear that it's not ready. And for a big patch like this, you're going to "start talking" long before actually submitting.
The only thing that supports your point is the incorrect title that was submitted to HN. But this submission got 6 upvotes and 0 comments, so almost nobody saw this title.
> Notably, Oracle does not include ZFS with Oracle Linux. As I said, they could easily clear this up, but they haven't.
> Oracle could relicense ZFS at any time, or make an explicit statement about its legal status. They have not done so after 10 years.
That would make this easy, but unfortunately easy is not the case we have here. Just as RH has chosen strict compliance with the GPL re RHEL, I think strict compliance with the GPLv2 is what is required here. Someone, I wish it was RH, should say "We've made a mistake. The GPL is not a suicide pact." For instance, interpreting any object which dynamically links to a GPL licensed work as a derived work is fundamentally anti-business, anti-interoperability, and, I'd argue, anti-FOSS. Being GPL compliant does not mean blithely accepting whatever legal claptrap FSF and SFC can summon.
There is a bunch of explanations and legal opinions here.
Oh? And let me tell you there is more than a boat load of lawyers who disagree.
See the links on page you linked to (!): http://www.networkworld.com/article/2301697/smb/encouraging-...
> There is a bunch of explanations and legal opinions here.
Did you read the text of that page or any of the linked articles? The OpenZFS position is clear: "However, there is nothing in either license that prevents distributing it in the form of a binary module or in the form of source code."
I guess that alone could be enough to come to the conclusion that it is not worth the trouble.
Linux had ~5 years to try and add ZFS before it was bought by Oracle, and did not do so. As much as Oracle is often the villain, I don't think they're what the Linux team fears here.
Frankly, it is more likely, distributions will add/ solidify infrastructure for making OpenZFS really hassle-free and well integrated and cloud/ VPS providers will have alternative images built with Root-on-ZFS. Until then, it is not that hard to convert a VM from EXT4 to OpenZFS or to build your own image. Shameless plug, I have done a video about a procedure how to convert EXT4 to OpenZFS from within the VM: https://www.youtube.com/watch?v=KoRVFRufZI0 Or you can just open the notes and copy&paste from there: https://github.com/KaliszAd/ext4-to-zfs-conversion
> And honestly, there is no way I can merge any of the ZFS efforts until I get an official letter from Oracle that is signed by their main legal counsel or preferably by Larry Ellison himself that says that yes, it's ok to do so and treat the end result as GPL'd.
> Other people think it can be ok to merge ZFS code into the kernel and that the module interface makes it ok, and that's their decision. But considering Oracle's litigious nature, and the questions over licensing, there's no way I can feel safe in ever doing so.
https://www.realworldtech.com/forum/?threadid=189711&curpost...
For me, ZFS beats other approaches with snapshots/ rollbacks and send/ receive. It really hits you, when you develop something using a database or a lot of files. If you want to test something, just do a snapshot (even without shutting down the database) and just try it out. If you mess up, you can at the very least shut the DB down, rollback (which is instant) and start the DB again. It will be in the state before you messed up. You can also make a clone and spin up another instance and compare. With files, you can even diff snapshots, so as to extract e.g. a deleted file. Also this is a great mitigation against ransomware, if your network drives are backed by ZFS. Also in the case of a database, compression and the integrity guarantees help a lot. You can do some tuning because of it that might make your performance better/ more consistent - because backups can get a lot more efficient and you can cache more data in RAM when it is compressed or buy better SSDs when they don't need to be that large. Now you can probably do something similar with btrfs, e.g. Turris Omnia is using btrfs internally and it saved me once while I was doing some broader reconfiguration. Otherwise, I don't know about it - it is probably reliable enough. You can do LVM snapshots. There are many approaches to solve a particular problem. ZFS was just designed to do some of these things and has about 15 years of production use in some of the most serious settings.
XFS or EXT4 is a decent choice for systems that you just don't want to think about. They are available by default, don't usually need any kind of tuning/ configuration. XFS is probably the better choice instead of EXT4, if you have a typical mdadm/ lvm RAID for e.g. surveillance camera recordings. For VMs that do some computation but the data is stored elsewhere, you can use the usually default EXT4 for the base filesystem. Converting such systems to ZFS might not be worth it. For the company database, filer, archive, backup ZFS should be something you evaluate as the first option. Just be sure you know a bit about how ZFS stores its data and how to configure the values for your use case. ZFS is more rigid in some places however extremely flexible in others be sure to understand where the rigid parts might be a problem to you.
My current/primitive understanding:
* CDDL is 'BSD-like'- allowing the right for CDDL code to co-exist in perhaps proprietary/closed source codebase, so long as the CDDL parts remain CDDL. It does not make demands for non-CDDL parts, allowing dual-license scenarios?
* GPL demands all code in the same codebase are subject to GPL. e.g. This excludes a scenario where some parts are open source, and others proprietary/closed.
* All contributions to the Linux kernel are subject to GPL
* Say OpenZFS is merged into Linux, CDDL demands that OpenZFS parts remain CDDL?
Now the resulting codebase has more restrictions than the spirit of the CDDL allows for. Is this not a conflict?
If you could attempt to ELI5 how I'm potentially getting part of this wrong, or point me to some article, I'd appreciate it! I honestly find this stuff pretty confusing.
> CDDL is 'BSD-like'- allowing the right for CDDL code to co-exist in perhaps proprietary/closed source codebase, so long as the CDDL parts remain CDDL. It does not make demands for non-CDDL parts, allowing dual-license scenarios?
It's more like halfway between GPL and BSD. Importantly, BSD licenses impose sufficiently limited conditions that BSD code can be subject to GPL conditions at the same time without violating either license. CDDL makes mutually exclusive demands with GPL.
(But yes, CDDL does allow for mixed CDDL+proprietary systems; that's what it was created for. Sun wanted to open source Solaris, but needed to be able to keep some parts of it closed-source (because they didn't actually own all the code, having licensed some of it from third parties).)
> All contributions to the Linux kernel are subject to GPL
There's actually non-GPL code in there, but it's all under GPL compatible licenses, so the final result is GPL. But for instance, if someone wanted to put in the work and there was interest, you could almost certainly get HAMMER2 from dragonfly BSD included into Linux because it'll be under a BSD license that's GPL compatible.
> Now the resulting codebase has more restrictions than the spirit of the CDDL allows for. Is this not a conflict?
Yes, that's basically the conflict. Note that there is some difference of opinion on the situation; Canonical, for instance, seems to believe that they can ship ZFS kernel modules under the CDDL without making them part of the GPL kernel and thus not mix the licenses. Others say that compiling the ZFS modules against the kernel makes the result a derivative work of the kernel and therefore is a conflict. Until someone actually tries to file a lawsuit over it, I don't know that we definitively know.
Suffice to say: a shallow reading of the GPLv2 might lead one to believe that the CDDL and the GPLv2 are incompatible. And I believe SFC and FSF's readings of the GPLv2 and copyright law, which the GPL incorporates by reference, are shallow. There are quite a few reasons to believe a CDDL and ZFS combination would not be incompatible. If you want learned legal opinions expressing my/this contrary view there are a few. [0], [1], [2]. Moreover, I'd warn you that SFC and FSF are not disinterested parties. They have agendas which extend beyond whether ZFS is actually incompatible to what that might mean for the GPL.
If you want my unschooled personal understanding -- most generally, there is no reason to believe that dynamic linking creates a derived work. Without further reasoning, FSF's analysis obliterates this boundary without reference to case law, and without an argument as why this must be the case. Whereas I think there are sound copyright law reasons to think where one creates an API boundary, through the use of dynamic linking or otherwise (a modular kernel interface), fair use allows one to make use of software at that boundary.
This is a difficult pill to swallow for many FOSS advocates because this would mean closed source software modules would be permissible in combination with the Linux kernel. And while I understand their political reasoning for disfavoring this view, I think their legal reasoning is weak. I also think the FSF view makes it much harder to build software which interoperates with other software, which BTW should be a key goal of the FSF and FOSS advocates.
I'm just not sure your copyright license has this kind of power. Imagine you build some software which links to MegaEvil Corp's libc, whose license specifically states, if you link to this libc, your work is a derived work of MegaCorp libc, you forfeit all copyright to MegaEvil Corp. My feeling is any court would say building any such software at an API boundary is fair use and copyright law simply does not grant any court the power to enforce this copyright term.
More specifically re: ZFS module and the GPL, I'd dig into what is a "derived work". The GPLv2 after all incorporates the copyright term by reference at Section 0 when describing a "work based on the Program". I'd consider the ambiguity of what is the "whole" work as described in Section 2 and how that might be construed in light of the long held copyright distinction between derived and collective works. I'd also consider recent copyright jurisprudence when asking whether any court would take the view that distributing the Linux kernel and ZFS module is a copyright violation. [3], [4].
[0]: https://softwarefreedom.org/resources/2016/linux-kernel-cddl... [1]: http://www.networkworld.com/article/2301697/smb/encouraging-... [2]: https://www.linuxjournal.com/article/6366 [3]: https://en.wikipedia.org/wiki/Sega_v._Accolade [4]: https://en.wikipedia.org/wiki/Lewis_Galoob_Toys,_Inc._v._Nin....
I just want to pause for a moment to explain how unhinged this reply is. First, never said Linux (or Oracle for that matter) was operating in bad faith. Second, my comment calls ZFS a "hair shirt" for the kernel. A "hair shirt" is an uncomfortable undergarment which religious people/GPL fanatics use to punish themselves.
Popped onto DDG just now, and the explains right there. Also "cilice" within the first few links.
After my unexpected laughter, I had to click. The first 2 hits align with my prior misunderstanding - cue more laughter. One is a t-shirt with a graphic of a very hairy chest/abdomen, the other has the likeness of Chewbacca with "Messy hair / don't care".
No need for anyone else to see for yourselves; saved you a click.
From: https://opensource.stackexchange.com/questions/2094/are-cddl...
> The CDDL is a license that does not try to enforce restrictions that would not stand in court but disallows relicensing of code. The fact that on the other side, the CDDL explicitly limits its scope to the files that contain CDDL'd code, allows us make our first conclusion:
> Code under the CDDL and code under the GPL cannot be mixed in a single file.
>The GPL is a license that limits its scope to the so called "work" limit and forbids relicensing of code. This allows us to make the next conclusion:
> Code under the CDDL and code under the GPL cannot be mixed in a single work.
So, that would seem to leave two options - either re-license Linux under a CDDL-compatible license or re-license ZFS under a GPL-compatible license. Clearly re-licensing ZFS would be much easier as it's controlled by a single entity - Oracle.
My opinion is that Oracle are acting in bad faith (probably a default assumption with Oracle to be fair) and deliberately don't want the code to be in Linux.
Oracle's ZFS code is controlled by a single entity, but OpenZFS (which is what you probably want) is a community fork with copyrights owned by a huge list of people and companies.
Edit: Ultimately unless it goes to court, all you can have are legal opinions - there is no legal authority.
In hindsight, an explicit "GPL compatibility clause" like the EUPL has would clearly been a wiser choice, but that wasn't clear in 2005, and it's too late for that now, unfortunately.
On the other hand, you could also blame the GPL for not recognizing that other copyleft licenses exist; they also could have written a compatibility clause in GPL 3. "CDDL is incompatible with the GPL" could also be phrased as "GPL is incompatible with the CDDL".
Yes, I agree; there's a few other topics like this. I call this "hacker conspiracy theories", because at this point that's basically what they are.
I mostly comment on these things for the benefit of innocent onlookers, who were not yet around when all of this went down, and not to convince the person who posted it.