Hacker News Reader

Comment by tivert | Hacker News Reader

> In the past few years the vast majority of new development in ZFS has taken place in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced that they will be moving to ZoL https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means that there will be little to no net new development of Illumos.

So, basically, Delphix was a (the only?) commercial user of Illumos and now they're abandoning it for Linux, which means Illumos will starve for manpower going forward?

It also sounds like FreeBSD also lacks the resources to maintain an implementation of something like ZFS, which is ironic since not too long ago it seemed like one of the marquee features they had that Linux didn't.

View on HN

> It also sounds like FreeBSD also lacks the resources to maintain an implementation of something like ZFS, which is ironic since not too long ago it seemed like one of the marquee features they had that Linux didn't.

There's no contradiction here. FreeBSD was never the primary contributor to (Open)ZFS (it was an Illumos thing), but we had the resources to make it a first class citizen on FreeBSD at a time when Linux wasn't doing that (mostly for licensing reasons).

It seems like the Linux world has gotten over the licensing problem one way or another, and at the end of the day Linux has something like 100x-1000x more developers to bring to bear at the problem. So between that, and illumos' imminent death, ZoL is the active upstream. FreeBSD is happy to benefit from the ZFS improvements in ZoL, especially given the collaborative spirit we've seen from the ZoL team so far. If anything, kudos to the ZoL folks for the huge amount of effort they've put in over the last handful of years.

I just wish the ZFS could be more tightly integrated into Linux than it can (due to licensing). Would be nice to not have two implementations of a crypto system, a raid parity computation system, block caching, allocation, etc. Oh well...

I hear you, especially on the block caching. ARC just does not interact well with FreeBSD's native block cache either.

The thing that really annoys me (if there is a way to fix this, please, comment) is that for zvols, whenever you read or write them, the blocks end up in both the ARC _and_ the Linux Page Cache.

is it just licensing? Doesn't it do its own thing in the vfs layer and perhaps others in a way that it wouldn't be accepted upstream either way?

I don't see any reason ZoL couldn't stub out their own CDDL implementations and invoke Linux functions directly, where they are a suitable replacement. The problem is almost always going to be that ZFS was designed for Solaris and adapting it to use Linux APIs is non-trivial. The code duplication bothers me less than the page cache duplication / cache reclaim logic integration.

It's not just to reduce the amount of ZoL specific work.

If I understand correctly, the developers were saying there are specific parts of Linux where anything using them is considered a derivative work, and therefore even distributing source would be a violation of GPL. An instance of this is the crypto layer that was added in recently with the native dataset encryption. SPL actually reimplements this logic because the ZoL maintainers claim they can't use the Linux crypto subsystem without violating GPL.

It does do various things which the Linux kernel maintainers don't like (particularly how tightly integrated all the components are, something something "rampant layering violation"). While getting ZFS in-tree would be a godsend, I don't mind building as an out-of-tree module, just that it would be nice if didn't duplicate so much functionality.

> illumos' imminent death

Rumours of our death are, of course, greatly exaggerated.

freebsd code base would be to be a marquee feature to me

I work at Joyent (one of the largest on-record commercial users of illumos today), and am a core member of the illumos community. While we will miss the Delphix folks, they were not driving the bulk of general work on illumos.

I believe that comment by Matt/Kip reflects more of a personal opinion about how things should be (he's a stalwart FreeBSD contributor), rather than how they actually are in practice.

Interesting. I took it to mean there wouldn't be much more ZFS development within the illumos community.

Huh, the rumors I've heard all say Joyent's use of illumos is not long for this earth.

So Bryan Cantrill (CTO) and all the former Sun eng. will abandon SmartOS @ Joyent?

I doubt that Joyent/Samsung will abandon their work on-top of Illumos. They contribute work upstream to Illumos.

They continue to port code to Illumos (i.e. KVM, continued work on Pluribus port of bhyve, etc.).

SmartOS is what their data centre orchestration solution gets built on-top of.

You can see their changelog for SmartOS they are quite active: https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/s...

Development of SmartOS and Triton Datacentre occur under their joyent organisation on GitHub.

The rumor is that the decision was made above that level.

Any links/sources you could point towards, even unofficial rumors?

No, just word of mouth.

I don't know what to tell you, except that people sometimes say things which aren't accurate when they believe it will advance their personal interests.

I can't disagree with that. :-)

iX is the corporate sponsor of FreeNAS, so it makes sense that they would have a stake in this, and are taking the lead on it. Hopefully this transition goes smoothly. I've been holding off on a new NAS for some of the migrations for FreeNAS to take effect.

On the flip side, I'm glad I saw this when I did. I really don't want to have to know all about ZFS tooling, and hoping that they get the UI/UX in place for a clean NAS experience, and may just buy a prebuilt from iX later next year.

I will complete my first freenas build that I've waited years to build this weekend. in either case, I've been waiting so long that seeing this does not change my desire and opportunity does not always come when we want it to.

guess I have some additionally learning to do :-)

>which is ironic since not too long ago it seemed like one of the marquee features they had that Linux didn't

That can still be somewhat true due to the licensing mess. ZoL will never be in the tree for linux. The best you can hope for is the Canonical solution which at least ensures a supported version for the distro kernel.

Unless and until Oracle relicenses ZFS in compatible terms.

My theory on how this might happen is an independent implementation of BtrFS that extends from ZFS code, sharing it whenever possible.

Should such a thing emerge and surpass current BtrFS, what else could Oracle do but throw in the towel?

Oracle is not capable of relicensing OpenZFS, as they do not hold the copyright for it past code that is common between ZFS v28/ZPOOL v5 and OpenZFS.

From my understanding both Linus and ZoL team prefer to be separate.

Yes but what if they Relicense the common code that was between ZFS, then the OpenZFS project should be able to relicense itself.

>From my understanding both Linus and ZoL team prefer to be separate.

I sincerely doubt that. If Oracle would relicense/dual license ZFS as GPLv2 compatible then I feel convinced that the ZoL contributors would happily do so as well.

I can't think of a single reason why they would not want to be in mainline Linux, and I have seen nothing indicating that the Linux kernel devs would be anything but happy to have it 'mainlined' as it would only make Linux better.

Of course this all hinges on Oracle, which makes it extremely unlikely in my opinion.

I can only base this on discussions with ZoL maintainers.

From my personal look, there could be significant mess due to attempts to "remove duplication" or similar moves to make the code more linux-like. While ZFS is not the rampant layering violation some people accused it of, there's significant amount of code there (and funnily enough, there's actually overlap with some less-known linux features)

Oracle is capable of changing the terms of the CDDL though (the CDDL has an auto-update clause), which would effectively allow for it to be replaced with a GPL-compatible version and it would affect all code put under that license (no matter who the copyright holder is). This is the same process that they used to switch from CDDLv1 to CDDLv2 for all their projects (though the only change was who had the right to change the license terms -- from Sun to Oracle).

But it's unlikely they'll do it. In recent years they've relicensed-under-GPL some things (like the DTrace userspace utilities) but I have doubts they'd ever do it for something like ZFS.

> Oracle is not capable of relicensing OpenZFS, as they do not hold the copyright for it past code that is common between ZFS v28/ZPOOL v5 and OpenZFS.

Yes, all the parties that contributed code would have to agree, not just Oracle. However, I doubt anyone would object relicensing ZFS to some license that both FreeBSD and Linux would find acceptable.

> It also sounds like FreeBSD also lacks the resources to maintain an implementation of something like ZFS

A bit of a leap - while your theory could be correct, alternatively, if the filesystem is complicated and can be ifdefed to work for many OS's, it makes sense to combine dev efforts into one codebase rather than having multiple independently maintained forks..

not sure which is the case here.. but in any event.

Reply on news.ycombinator.com