Hacker News Reader

Comment by thyristan | Hacker News Reader

Both are imho linked to "offline desktop use cases are not important anymore". Both companies saw their future gains elsewhere, in internet-related functions and what became known as "cloud". No need to have a fancy, featurefull and expensive filesystem when it is only to be used as a cache for remote cloud stuff.

View on HN

Internet connections of the day didn't yet offer enough speed for cloud storage.

Apple was already working to integrate ZFS when Oracle bought Sun.

From TFA:

> ZFS was featured in the keynotes, it was on the developer disc handed out to attendees, and it was even mentioned on the Mac OS X Server website. Apple had been working on its port since 2006 and now it was functional enough to be put on full display.

However, once Oracle bought Sun, the deal was off.

Again from TFA:

> The Apple-ZFS deal was brought for Larry Ellison's approval, the first-born child of the conquered land brought to be blessed by the new king. "I'll tell you about doing business with my best friend Steve Jobs," he apparently said, "I don't do business with my best friend Steve Jobs."

And that was the end.

Was it not open source at that point?

It was! And Apple seemed fine with including DTrace under the CDDL. I’m not sure why Apple wanted some additional arrangement but they did.

The NetApp lawsuit. Apple wanted indemnification, and Sun/Oracle did not want to indemnify Apple.

At the time that NetApp filed its lawsuit I blogged about how ZFS was a straightforward evolution of BSD 4.4's log structured filesystem. I didn't know that to be the case historically, that is, I didn't know if Bonwick was inspired by LFS, but I showed how almost in every way ZFS was a simple evolution of LFS. I showed my blog to Jeff to see how he felt about it, and he didn't say much but he did acknowledge it. The point of that blog was to show that there was prior art and that NetApp's lawsuit was worthless. I pointed it out to Sun's general counsel, too.

While I was disappointed that NetApp sued, the ZFS team literally referenced NetApp and WAFL multiple times in their presenations IIRC. They were kind of begging to be sued.

Also, according to NetApp, "Sun started it".

https://www.networkcomputing.com/data-center-networking/neta...

No, the ZFS team did not "literally reference NetApp and WAFL" in their presentations and no, Sun did not "start it" -- NetApp initiated the litigation (though Sun absolutely countersued), and NetApp were well on their way to losing not only their case but also their WAFL patents when Oracle acquired Sun. Despite having inherited a winning case, Oracle chose to allow the suit to be dismissed[0]; terms of the settlement were undisclosed.

[0] https://www.theregister.com/2010/09/09/oracle_netapp_zfs_dis...

We can agree to disagree.

Your own link states that Sun approached NetApp about patents 18 months prior to the lawsuit being filed (to be clear that was Storagetek before Sun acquired them):

>The suit was filed in September 2007, in Texas, three years ago, but the spat between the two started 18 months before that, according to NetApp, when Sun's lawyers contacted NetApp saying its products violated Sun patents, and requesting licensing agreements and royalties for the technologies concerned.

And there was a copy of the original email from the lawyer which I sadly did not save a copy of, as referenced here:

https://ntptest.typepad.com/dave/2007/09/sun-patent-team.htm...

As for the presentation, I can't find it at the moment but will keep looking because I do remember it. That being said, a blog post from Val at the time specifically mentions NetApp, WAFL, how the team thought it was cool and decided to build your own:

https://web.archive.org/web/20051231160415/http://blogs.sun....

And the original paper on ZFS that appears to have been scrubbed from the internet mentions WAFL repeatedly (and you were a co-author so I'm not sure why you're saying you didn't reference NetApp or WAFL):

https://ntptest.typepad.com/dave/2007/09/netapp-sues-sun.htm...

https://www.academia.edu/20291242/Zfs_overview

>The file system that has come closest to our design principles, other than ZFS itself,is WAFL[8],the file system used internally by Network Appliance’s NFS server appliances.

> And the original paper on ZFS that appears to have been scrubbed from the internet mentions WAFL repeatedly (and you were a co-author so I'm not sure why you're saying you didn't reference NetApp or WAFL):

Cantrill was not involved in ZFS, and was not a co-author. Cantrill was involved with DTrace:

* https://www.usenix.org/conference/2004-usenix-annual-technic...

* https://www.cs.princeton.edu/courses/archive/fall05/cos518/p...

And the ZFS paper has hardly been scrubbed given it is widely cited:

* https://www.cs.hmc.edu/~rhodes/cs134/readings/The%20Zettabyt...

And the fact that the ZFS paper cites WAFL is hardly an indication of anything, given that NetApp's patent cites a whole bunch of other patents:

* https://patents.google.com/patent/US5819292#patentCitations

Heck, some of the cited patents were Sun's.

* https://en.wikipedia.org/wiki/NetApp#Legal_dispute_with_Sun_...

> The file system that has come closest to our design principles, other than ZFS itself,is WAFL[8],the file system used internally by Network Appliance’s NFS server appliances.

That was unnecessary, but that does not betray even the slightest risk of violating NetApp's patents. It just brings attention.

Also, it's not true! The BSD 4.4 log-structured filesystem is such a close analog to ZFS that I think it's clear that it "has come closest to our design principles". I guess Bonwick et. al. were not really aware of LFS. Sad.

LFS had:

  - "write anywhere"
  - "inode file"
  - copy on write

LFS did not have:

  - checksumming
  - snapshots and cloning
  - volume management

And the free space management story on LFS was incomplete.

So ZFS can be seen as adding to LFS these things:

  - checksumming
  - birth transaction IDs
  - snapshots, cloning, and later dedup
  - proper free space management
  - volume management, vdevs, raidz

I'm not familiar enough with WAFL to say how much overlap there is with WAFL, but I know that LFS long predates WAFL and ZFS. LFS was prior art! Plus there was lots of literature on copy-on-write b-trees and such in the 80s, so there was lots of prior art in that space.

Even content-addressed storage (CAS) (which ZFS isn't quite) had prior art.

> I guess Bonwick et. al. were not really aware of LFS. Sad.

They were:

> [16] Mendel Rosenblum and John K. Ousterhout. The design and implementation of a log-structured file system. ACM Transactions on Computer Systems, 10(1):26–52, 1992.

> [17] Margo Seltzer, Keith Bostic, Marshall K. McKusick, and Carl Staelin. An implementation of a log-structured file system for UNIX. In Proceedings of the 1993 USENIX Winter Technical Conference, 1993.

* https://www.cs.hmc.edu/~rhodes/cs134/readings/The%20Zettabyt...

Seems specious. Patents don't preclude one from overtly trying to compete; they protect specific mechanisms. In this case either ZFS didn't use the same mechanisms or the mechanisms themselves were found to have prior art.

Whether the claims were valid or not I guess we'll never know given Oracle and NetApp decided to settle.

What I DO knows is that if the non-infringement were as open and shut as you and Bryan are suggesting, Apple probably wouldn't have scrapped years of effort and likely millions in R&D for no reason. It's not like they couldn't afford some lawyers to defend a frivelous lawsuit...

Maybe! Bryan and I were pretty close to the case and to the implementation of ZFS. But maybe Apple did detect some smoking gun of which somehow we were unaware. I (still) think Jonathan’s preannouncement was the catalyst for Apple changing direction.

I will just say I can’t thank both of you enough. I cut my teeth on zfs and it was a pillar of the rest of my career.

It’s a constant reminder to me of the value of giving that college kid free access to your code so they can become the next expert doing something creative you never thought of.

There was lots of prior art from the 80s for "write anywhere", which is a generally a consequence of copy-on-write on-disk formats. The write-anywhere thing is not really what's interesting, but, rather, not having to commit to some number of inodes at newfs time. Sun talking about NetApp makes sense given that they were the competition.

We don't know exactly what happened with Apple and Sun, but there were lots of indicia that Apple wanted indemnification and Sun was unwilling to go there. Why Apple really insisted on that, I don't know -- I think they should have been able to do the prior art search and know that NetApp probably wouldn't win their lawsuits, but hey, lawsuits are a somewhat random function and I guess Apple didn't want NetApp holding them by the short ones. DTrace they could remove, but removing ZFS once they were reliant on it would be much much harder.

Ok? So what?

The argument advanced in the piece isn't without merit -- that ripping out DTrace, if subsequent legal developments demanded it, would be a heck of a lot easier than removing a filesystem that would by then contain massive amounts of customer data.

And given what a litigious jackass Larry Ellison / Oracle is, I can't fault Apple for being nervous.

No, it was that Apple wanted to be indemnified in the event that Sun/Oracle lost the NetApp lawsuit.

Ironic since the post above tells the story as LE saying no to Jobs.

There's always two sides to one story. And Jobs was not the kind of person to take no for an answer. I've known other narcissists and they've always seen friends as more of a resource than someone to care about.

I think the truth is somewhere in the middle.

Another rumour was that Schwartz spilling the beans pissed Jobs off, which I wouldn't really put past him. Though I don't think it would have been enough to kill this.

I think all these little things added up and the end result was just "better not then".

Sure, but what sane person would not expect Oracle to continue to be litigious?

I imagine the situation would have been different if Apple's ZFS integration had completed and shipped before Sun's demise.

They didn't rip out DTrace, after all.

Exactly this.

The business case for providing a robust desktop filesystem simply doesn’t exist anymore.

20 years ago, (regular) people stored their data on computers and those needed to be dependable. Phones existed, but not to the extent they do today.

Fast forward 20 years, and many people don’t even own a computer (in the traditional sense, many have consoles). People now have their entire life on their phones, backed up and/or stored in the cloud.

SSDs also became “large enough” that HDDs are mostly a thing of the past in consumer computers.

Instead you today have high reliability hardware and software in the cloud, which arguably is much more resilient than anything you could reasonably cook up at home. Besides the hardware (power, internet, fire suppression, physical security, etc), you’re also typically looking at multi geographical redundancy across multiple data centers using reed-Solomon erasure coding, but that’s nothing the ordinary user needs to know about.

Most cloud services also offer some kind of snapshot functionality as malware protection (ie OneDrive offers unlimited snapshots for 30 days rolling).

Truth is that most people are way better off just storing their data in the cloud and making a backup at home, though many people seem to ignore the latter, and Apple makes it exceptionally hard to automate.

What do you do when you discover that some thing you have not touched in a long time, but suddenly need, is corrupted and all of your backups are corrupt because the corruption happened prior to your 30 day window at OneDrive?

You would have early warning with ZFS. You have data loss with your plan.

You should have bought OneDrive Premium Plus with Extended Backup Storage (tm). This is your fault for not shelling out, stop blaming your computer.

You do realise that in your use cases you still need reliable filesystems both in the cloud and on devices.

Because any corruption at any point will get synced as a change, or worse can cause failure.

Not with any proper erasure coded distributed filesystem, you just trash and replace the entire drive and let ceph or whatever distributed storage solution you use rebalance the shards from other nodes. You don't need the filesystem to be reliable on a device level, you just need basic checksums for integrity checking and enough parity on the cluster level to replace bad drives.

Workstation use cases exist. Data archival is not the only application of file systems.

I remember eagerly anticipating ZFS for desktop hard disks. I seem to remember it never took off because memory requirements were too high and payoffs were insufficient to justify the trade off.

Linux or FreeBSD developers are free to adopt ZFS as their primary file systems. But it appears that practical benefits are not really evident to most users.

Lots of ZFS users are enthusiasts who heard about that one magic thing that does it all in one tidy box. Whereas usually you would have to known all the minutiae of LVM/mdadm/cryptsetup/nbd and mkfs.whatever to get to the same point. So while ZFS is the nicer-dicer of volume management and filesystems, the latter is your whole chef's knife set. And while you can dice with both, the user groups are not the same. And enthusiasts with the right usecases are very few.

And for the thin-provisioned snapshotted subvolume usecase, btrfs is currently eating ZFS's lunch due to far better Linux integration. Think snapshots at every update, and having a/b boot to get back to a known-working config after an update. So widespread adoption through the distro route is out of the question.

Ubuntu's ZFS-on-root with zsys auto snapshots have been working excellently on my server for 5 years. It automatically takes snapshots on every update and adds entries to grub so rolling back to the last good state is just a reboot away.

> And for the thin-provisioned snapshotted subvolume usecase, btrfs is currently eating ZFS's lunch due to far better Linux integration.

Is this a technical argument? Or is this just more licensing nonsense?

> Think snapshots at every update, and having a/b boot to get back to a known-working config after an update.

I use ZFS and I have snapshots on every update? I have snapshots hourly, daily, weekly and monthly. I have triggered snapshots too, and ad hoc dynamic snapshots too. I wrote about it here: https://kimono-koans.github.io/opinionated-guide/

> Think snapshots at every update, and having a/b boot to get back to a known-working config after an update.

Solaris had this about a decade ago with beadm:

* https://docs.oracle.com/cd/E53394_01/html/E54749/gpxnl.html

FreeBSD has had it for several years as well. If Linux lacks it that is a failure of Linux (distros) not ZFS.

The ZFS license makes it impossible to include in upstream Linux kernel, which makes it much less usable as primary filesystem.

Linux’s signed off policy makes that impossible. Linus Torvalds would need Larry Elison’s signed off before even considering it. Linus told me this by email around 2013 (if I recall correctly) when I emailed him to discuss user requests for upstream inclusion. He had no concerns about the license being different at the time.

That's called marketing. Give it a snazzy name, like say "TimeMachine" and users will jump on it.

Also, ZFS has a bad name within the Linux community due to some licensing stuff. I find that most BSD users don't really care about such legalese and most people I know that run FreeBSD are running ZFS on root. Which works amazingly well I might add.

Especially with something like sanoid added to it, it basically does the same as timemachine on mac, a feature that users love. Albeit stored on the same drive (but with syncoid or just manually rolled zfs send/recv scripts you can do that on another location too).

> I find that most BSD users don't really care about such legalese and most people I know that run FreeBSD are running ZFS on root.

I don't think it's that they don't care, it's that the CDDL and BSD-ish licenses are generally believed to just not have the conflict that CDDL and GPL might. (IANAL, make your own conclusions about whether either of those are true)

Hmm yeah but that's the thing, who really cares about licenses as a user? I certainly don't. It's just some stuff that some lawyers fuss over. I don't read EULAs either nor would I even consider obeying them. The whole civil-legal world is just something I ignore.

I do have a feeling that Linux users in general care more about the GPL which is quite specific of course. Though I wonder if anyone chooses Linux for that reason.

But really personally I don't care whether companies give anything back, if anything I would love less corporate involvement in the OS I use. It was one of my main reasons for picking BSD. The others were a less fragmented ecosystem and less push to change things constantly.

I feel the same way. The way people talk about how ZFS is incompatible with Linux feels like debates over religious doctrine.

> ZFS has a bad name within the Linux community due to some licensing stuff

This is out of an abundance of caution. Canonical bundle ZFS in the Ubuntu kernel and no one sued them (yet).

True and I understand the caution considering Oracle is involved which are an awful company to do deal with (and their takeover of Sun was a disaster).

But really, this is a concern for distros. Not for end users. Yet many of the Linux users I speak to are somehow worried about this. Most can't even describe the provisions of the GPL so I don't really know what that's about. Just something they picked up, I guess.

Licensing concerns that prevent distros from using ZFS will sooner or later also have adverse effects on end users. Actually those effects are already there: The constant need to adapt a large patchset to the current kernel, meaning updates are a hassle. The lack of packaging in distributions, meaning updates are a hassle. And the lack of integration and related tooling, meaning many features can not be used (like a/b boots from snapshots after updates) easily, and installers won't know about ZFS so you have to install manually.

None of this is a worry about being sued as an end user. But all of those are worries that you life will be harder with ZFS, and a lot harder as soon as the first lawsuits hit anyone, because all the current (small) efforts to keep it working will cease immediately.

Unlike other out of tree filesystems such as Reiser4, the ZFS driver does not patch the kernel sources.

That is due to licensing reasons, yes. It makes maintaining the codebase even more complicated because when the kernel module API changes (which it very frequently does) you cannot just adapt it to your needs, you have to work around all the new changes that are there in the new version.

You have things backward. Licensing has nothing to do with it. Changes to the kernel are unnecessary. Maintaining the code base is also simplified by supporting the various kernel versions the way that they are currently supported.

Time Machine was released 17 years ago, and I wish Windows had anything that good. And they're on their 3rd backup system since then.

Windows has a really good basis for it though, in volume shadow copy. I also don't understand why Microsoft never built a time machine based on that. Well, they kinda did but only on samba shares. But not locally.

But these days they want you to subscribe to their cloud storage so the versioning is done there, which makes sense in their commercial point of view.

I think snapshots on ZFS are better than time machine though. Time machine is a bit of a clunky mess of soft links that can really go to shit on a minor corruption. Leaving you with an unrestorable backup and just some vague error messages.

I worked a lot with macs and I've had my share of bad backups when trying to fix people's problems. I've not seen ZFS fail like that. It's really solid and tends to indicate issues before they lead to bigger problems.

Hard links[0].

[0] https://en.wikipedia.org/wiki/Time_Machine_(macOS)#Operation

Hard links are only used on HFS+. APFS has snapshot support.

Shadow Protect by StorageCraft was brilliant. Pretty sure MS actually licensed Shadow Copy from them, but I could be mistaken. It's been a while since I played in that space.

Time Machine is good for the technically savvy, but for the non-tech-savvy, without something like the AirPort Time Capsule there's no real easy configuration for it.

> I wish Windows had anything that good

I can't readily tell how much of the dumbness is from the filesystem and how much from the kernel but the end result is that until it gets away from 1980s version of file locking there's no prayer. Imagine having to explain to your boss that your .docx wasn't backed up because you left Word open over the weekend. A just catastrophically idiotic design

Ah but this is really not true. Volume shadow copy makes snapshots of files and through that it can make a backup of an entire NTFS including files with a lock on them and fully quiesced. It was invented for that exact purpose. Backup software on windows leverages this functionality well. It took much longer for Linux to have something similar.

I have many criticisms of NTFS like it being really bad at handling large volumes of small files. But this is something it can do well.

The lock prevents other people from copying the file or opening it even in read only, yes. But backup software can back it up just fine.

>I find that most BSD users don't really care about such legalese and most people I know that run FreeBSD are running ZFS on root.

What a weird take. BSD's license is compatible with ZFS, that's why. "Don't really care?" Really? Come on.

There is an increase of posts with casual confidence in their own absolute correctnes. I originally attributed it to the influence of llms, but it is becoming so common now it is hard to dismiss.

I never claimed absolute correctness, just stating what I see with the other BSD users I know.

You did say that a most linux admins dont really care.. which has been edited out now.

Huh no I never spoke of Linux users

Did i respond to the wrong thread then? I'm pretty sure there was context around "linux users dont really seem to care about zfs licencing".

Are you saying you didnt edit the post ?

I just mean that GPL is a bit of a religion. There are very strong opinions and principles behind it. Whereas the BSD license is more like "do whatever you want". It makes sense that the followers of the former care more deeply about it, right?

Personally I don't care about or obey any software licenses, as a user.

But this is kinda the vibe I get from other BSD users if a license discussion comes up. Maybe it's my bubble, that's possible.

I have to strongly disagree with you that GPL is a religion. While there Are certainly people who sound religious in nature when they talk about the GPL, The real reason it is important is because of what it forces developers to do on behalf of the users. It balances somewhat the power between the developer and the user.

Under other licensing, developers wield an extraordinary amount of power over the users. Yes, The user could opt not to run that code, but realistically that isn't an option in the modern day. Developers can and will abuse their access to your machine to serve their ends regardless of whether it adds value to you or not. For example, how much data collection is in nearly all modern software?

Perhaps you would argue that what I've said above only applies to a very tiny minority of users who have the technical skills to actually utilize the code, and everyone else It's just a religious argument. I don't fully disagree with that. There is another clear benefit That even those untechnical users received from the GPL, and that is the essentially forced contribution back from companies who want to build on top of it. I don't think there's any better example than the Linux kernel, which has gotten lots of contributions from companies that are otherwise very proprietary in nature and would never have open sourced things. This has benefited everyone and has acted as a rising tide lifting All boats. Without the requirements in the GPL, this most certainly would not happen.

My response to it however, is that those users still get a good amount of protection because The code is out there

Oops that last sentence (which is a sentence fragment) was supposed to be deleted, but slipped in some how and it's too late to edit. The point I was going to make was just that with the code being out there, the odds that some offensive thing the devs might do can be discovered by someone and have the issue raised. It also provides a powerful incentive to not stick something gross in there for risk of it being discovered and getting called out for it :-)

I get what you mean and probably there are people like that, but I consider it mostly an exaggeration.

Simply - the GPL has some clauses enforcing some obligations (to prevent some rights from being taken away from you, the end user - according to their wording, and I agree), these and other clauses make it legally incompatible with the inclusion of ZFS (CDDL license) in the Linux kernel (GPL). You can build it yourself (so indeed as a user you get to not care or obey) but not distribute it (this is the problem of your distribution's maintainer).

Canonical's lawyers think this is not a problem if the ZFS code is distributed as a module, instead of compiled into the kernel itself, and since 2016 Ubuntu shipped with ZFS support.

The BSD license is considered perfectly compatible with the inclusion of CDDL licensed code and therefore many BSD distros ship with ZFS (and Dtrace) out of the box without legal worries. Indeed Oracle hasn't come knocking.

TL;DR: it's not a vibe. Some licenses are compatible with each other, some aren't. It also depends on how different licenses come into play into a "finished product" (e.g. kernel module vs monolithic build)

ZFS on FreeBSD is quite nice. System tools like freebsd-update integrate well. UFS continues to work as well, and may be more appropriate for some use cases where ZFS isn't a good fit, copy on write is sometimes very expensive.

Afaik, the FreeBSD position is both ZFS and UFS are fully supported and neither is secondary to the other; the installer asks what you want from ZFS, UFS, Manual (with a menu based tool), or Shell and you do whatever; in that order, so maybe a slight preferance towards ZFS.

OpenZFS exists and there is a port of it for Mac OS X.

The problem is that it is still owned by Oracle. And Solaris ZFS is incompatible with OpenZFS. Not that people really use Solaris anymore.

It is really unfortunate. Linux has adopted file systems from other operating systems before. It is just nobody trust Oracle.

ZFS is a first class citizen in FreeBSD and has been for at least a decade(probably longer). Not at all like in most Linux distros.

Reply on news.ycombinator.com