Hacker News Reader

Comment by justin66 | Hacker News Reader

I am as happy as anyone that XFS is finally getting the position of honor it deserves on enterprise Linux (something like 15 years later than it should have, grumble grumble) but it doesn't really take the place of what btrfs was trying to do. Only ZFS is in a position to do that. I wonder if there are any plans for supporting the native port on RHEL.

View on HN

Anecdote time: Last month I had an XFS volume fail on me. It got some sort of internal inconsistency and refused to work (all fs calls returned errors until I unmounted). This is where I discovered that XFS still has extremely poor recovery tools.

xfs_repair will complain if there is a journal present and tell you to mount the fs to replay the journal. But mount would refuse, saying the fs was inconsistent. So the only option was to xfs_repair -L to just throw out the journal.

Then, xfs_repair sucked up something like 30GB or more of RAM, so I had to make a huge swapfile so that the kernel would OOM the repair.

Then, after roughly 20 to 30 hours of repair it would exit with an error. At that point it would actually mount, but hitting certain areas of the filesystem would trigger the inconsistency again and start the entire process over.

In the end I couldn't fix it and sadly had to reformat. I chose ext4 when I did—I've had lots of experience with ext3 and 4 and I've never had a filesystem that I couldn't at least make consistent again (even if it loses some data).

Yikes. That's unnerving to read. Were you able to create a bug report? Was there any other related issue like an underlying storage controller messing up?

Redhat hasn't been on the best of terms with Oracle, so I suspect that they want to stay clear of ZFS. It does however leave Redhat without a more modern feature rich filesystem.

Perhaps Redhat could help to develop snapshots on XFS. It's not the only feature XFS is missing, but it's a start.

> Perhaps Redhat could help to develop snapshots on XFS. It's not the only feature XFS is missing, but it's a start.

Upstream has been working on adding more btrfs-like features to XFS, but I believe that RHEL encourages using devicemapper snapshots (which you then format with XFS).

> Upstream has been working on adding more btrfs-like features to XFS, but I believe that RHEL encourages using devicemapper snapshots (which you then format with XFS).

Exactly, mountable and mergable snapshots have been supported by a LVM/devicemapper stack for a long time.

It's still a bit more involved than the transparent snapshots ZFS and, to a lesser degree, BtrFS offer. I'm not happy with this and sincerely hope this position changes in the future.

>Redhat hasn't been on the best of terms with Oracle, so I suspect that they want to stay clear of ZFS

The only connection Oracle has to ZFS on Linux is ownership of some patents that the license allows you to use, their reluctance is based on distribution issues between the GPL and CDDL.

… for which Oracle owns the copyright.

I can't see that the copyright matter here. What matters is the license. Oracle can't unlicence what Sun code is already part of OpenZFS.

But presumably, as copyright holders, Oracle is the entity that could try to enforce CDDL in court, in particular breaking the CDDL by mixing in GPL code in the same (ie: OS) distribution? Oracle goes: we bought Sun, and hold copyright to ZFS (also at the point of the OpenZFS fork) - RedHat could respond: we got a license - the CDDL - and Oracle could respond, sure - but CDDL isn't compatible with GPL - so you're in breach of the CDDL?

>But presumably, as copyright holders, Oracle is the entity that could try to enforce CDDL in court, in particular breaking the CDDL by mixing in GPL code in the same (ie: OS) distribution?

Any Linux contributor could also try to enforce it, which is why the license incompatibility is the issue stopping them. Oracle holds no special power.

True - but the incentives are a bit different. How many other Linux contributors[1] are selling a commercial operating system in direct competition with Linux as a general purpose Unix-like OS, with ZFS as one of the differentiating features?

Most Linux contributors want Linux to succeed. I don't think it's at all clear that corporate Oracle prefers Linux to succeed - at least not if higher adoption of Solaris is an alternative.

[1] (I guess IBM and Microsoft come to mind... but they don't have any special investment in ZFS)

There are thousands of Linux contributors, I don't care enough to check but I have to imagine Oracle has employed at least one. Any one of them could sue, and several have mentioned they're considering the option.

The license issue is what's keeping Red Hat from using ZFS, not some rivalry with Oracle.

How does mixing CDDL and GPL violate CDDL?

The only issue I'm aware is that mixing the two would violate GPL.

CDDL says: Source code must be licensed under CDDL.

GPL says: Source code must be licensed under GPL.

If you follow the conditions of GPL, you are violating the condition of the CDDL. If you are following the conditions of CDDL, you are violating the GPL. Basic binary logic.

To add: "the engineers who had written the Solaris kernel requested that the license of OpenSolaris be GPL-incompatible". A license is really just an written intention of the author on what conditions copyright law restrictions may be legally ignored. In this case, those wishes had a very explicit intention. However those using the license today has had a general change of heart, and those with GPL interest has a general stance that no FOSS project will ever sue an other FOSS project over license incompatibility. As such, the risk of lawsuit is really just a company suing an other company under the technicality of incompatibility.

Naturally some organizations won't intentionally break copyright law just because no one will sue.

Neither license forbids mixing with other licenses. As long as the demands of both are met, they can apply to the same source code.

>If you are following the conditions of CDDL, you are violating the GPL. Basic binary logic.

Relationship between licenses can be transitive but not commutative.

As far as I know CDDL allows using with code under GPL but GPL does not allow using code under CDDL. CDDL copyright owners have no case, GPL copyright owners have.

The question is: If I'm incorrect, what in CDDL prevents using with GPL?

If CDDL has no issue with GPL conditions, then follow the GPL and everything is fine.

CDDL has this text: "Any Covered Software that You distribute or otherwise make available in Executable form must also be made available in Source Code form and that Source Code form must be distributed only under the terms of this License"

So you take some CDDL code, and some GPL code, and you put that whole new source code tree under GPL in order to fullfill the GPL license condition. Are you then in compliance with the CDDL code? My concussion is that you are not, as that would be in conflict with the above condition of the CDDL. The source code tree would not be "distributed only under the terms of this license".

CDDL is per-file.

How does this change this?

I take a CDDL licensed source code file. I take a GPL licensed source code file. I add inline the GPL licensed code to the CDDL licensed file, and release an executable form of the result. In order to comply with the GPL I then give out a single source code file under the GPL license terms with the code from the two files.

Is this in compliance with the CDDL terms and conditions?

This stackoverflow question have some good points (notably, the accepted answer, and the bit about limitations - the CDDL section 6.2, for example revokes the CDDL in case of patent infringement, something that might be considered an "extra limitation" under the GPL (you're not allowed to add additional limitations to either the GPL or the CDDL). As such, the CDDL might be incompatible with the GPL up to and including v2 - while GPL3 might also be incompatible with the CDDL):

https://opensource.stackexchange.com/questions/2094/are-cddl...

https://github.com/zfsonlinux/zfs/blob/master/OPENSOLARIS.LI...

Also, the SO answer mentions consumer protection laws - but AFAIK they generally only apply to consumers - not businesses. So the GPL 0 clause might be void in many jurisdictions for individuals but still valid for businesses.

Oracle is not the one that could sue. There is nothing in the CDDL that prevents it being used else where.

The GPL on the other hand is a strong copy left. If you link against GPL code, your code must also be licensed as GPL.

This means the Linux copyright owners could sue the distributers of ZoL binaries, but Oracle could not.

Oracle has the power to allow their ZFS code to be relicensed as GPL, removing this road block, but they have no incentive to do so.

Correct, Oracle would be in a position to do exactly that. Whether they do or not is another story but thats a pretty big liability.

Given they are discontinuing Solaris and all-in on Red Hat Enterprise Linux I can't help but wonder why they don't do more with ZFS on Linux and therefor wonder if the NetApp patent suits or some other patent suit is preventing them from doing anything in the background.

Many people don't realise that these crappy patent suits in the background prevent all sorts of really basic stuff, like the fact most things now bounce through a cloud server (like Facetime) because there's a patent troll for peer-to-peer communications. And it's causing total waste as a result :( It also seems likely that prevented facetime becoming an open standard as Apple original promised. This is only 1 example though.

> Given they are discontinuing Solaris

Oracle is NOT discontinuing Solaris. This FUD must die.

https://news.ycombinator.com/item?id=14865237

This may be true, but when Oracle killed OpenSolaris, my non-Sun/Oracle friends wrote off Solaris and moved off it.

Killing OpenSolaris, and talking up SPARC so much, made people think that a) Larry just wants to vendor lock them, b) doesn't care about x86 support because it makes vendor lock-in harder for Oracle, c) the OpenSolaris derivative community will not be able to compete with Linux. So everyone has grudgingly accepted that Linux is it for the enterprise Unix market.

I hate this as much as you do. I <3 Solaris/Illumos. Illumos derivatives have their niches, no doubt, and I want to be able to use them much more. But that's not how business people think.

I'm not sure that Oracle could turn this impression around at this point. To begin with it would have to restart OpenSolaris, and that might not be enough. OpenSolaris greatly helped Sun overcome resistance to Solaris, but it only went so far, so Oracle will have to do even more work to make Solaris' future bright.

This blog post is as relevant today as ever: https://blogs.oracle.com/bmc/the-economics-of-software

(And yes, it's STILL hosted at blogs.oracle.com. I'm almost afraid of mentioning it: who knows, it might get removed if Oracle execs notice it.)

Hrm thanks seems you're right. They cancelled Solaris "12" but 11 is still in development. Thanks for the correction.

https://theregister.co.uk/2017/01/18/solaris_12_disappears_f...

Update: Apparently Oracle actually wouldn't be the liability on a CDDL basis because the violation is of the GPL not the CDL. Fair point.

Oracle can relicense their codebase anytime they want. They _cannot_ be constrained by OpenZFS/Illumos unless they accept patches from them without copyright assignment.

What about just using LVM for snapshots? Considering that there default partition schemes include LVM maybe that's what they bank on in most use cases?

I have one machine running XFS but if that one is representative then I won't be installing XFS anywhere else and would happily discourage others from using it. It is terribly slow when doing some fairly common operations when you have a large number of small files.

You're joking, surely.

XFS has outperformed EXT4 in almost all "high" use-cases in my experience and testing: Large files (500GB~) or many small files (128k files * 2,400,000 or so). EXT4 under those loads is comically bad.

BTRFS is also terrible at this, only XFS and ZFS are good at handling it.

On the other hand on database workloads, for example PostgreSQL, XFS and EXT4 are about equal these days. ZFS (at least on Linux) and Btrfs are both clearly slower on those workloads.

Here is one benchmark, but I have seen plenty of similar benchmark results for PostgreSQL showing the same thing: https://blog.pgaddict.com/posts/postgresql-performance-on-ex...

There's a fairly simple reason for that which is that ZFS (and btrfs to some extent) are almost literally "ACID" databases. They do alot of the same double-writes and other safe behaviour the database is also doing. Those have a penalty and you're doubled up.

There are various guides around for tuning ZFS and database servers to try reduce that duplication, for example you can disable the InnoDB double write buffer because ZFS guarantees you don't need it. You also need to tune recordsize to match the database page size so that you don't accidentally create large multi page blocks.

I partially disagree with the claim that ZFS is slower than ext4/xfs. It is, but only as long as you don't use ext4/xfs on top of LVM to get similar snapshot capabilities etc. Then ZFS starts to win.

So if you only need a plain filesystem, ext4/xfs are great and you will get better performance.

If you need/want snapshots, e.g. to do backups that way, it makes sense to look at ZFS.

I wish I was. It's pathetic. Creating a new directory entry on an idle machine with plenty of CPU and memory takes seconds, ditto deletions.

Also: I love how that comment sits at -4, as if downvoting it will somehow discredit the data point.

> I have one machine running XFS but if that one is representative

> Creating a new directory entry on an idle machine with plenty of CPU and memory takes seconds, ditto deletions.

I think your answer lies in your premise then. It's not representative.

10's millions of files should have been the ideal use case for XFS, that's why I installed it in the first place. This was for the 'reocities.com' project and by the time I realized what the problem was most of the import had already been done so I let it run to completion but it makes updating the project a real PITA.

Sure, but the issue could be configuration, drive, interface, etc. It's impossible to speculate in, but what we know is you have trouble with one machine, and it's the only one that has used XFS. It's unfortunate, but likely a coincidence, or at least unrelated to XFS at its core.

I've been using XFS for 10 years without the issues you seem to be having.

There's so much that can go wrong setting up a Linux server that it's impossible to give much advice with something like this.

I guess the general stuff is: the easy default partitioning setup you get from a Linux distro is total bs, you need more RAM than you think you do, the way you're serving files or accessing the system (NFS!) has plenty of ways to screw things up as well, and tens or hundreds of millions of files is not any filesystem's ideal use case. The classic IRIX workload would be guaranteed-rate streaming of large media files, and the Linux port of the filesystem obviously inherited a lot of that system's traits (without the GRIO).

XFS has received some very serious performance improvements in the past couple of years to address indexing, large volumes of metadata, and so on, so that'd be one very relevant thing. Dave Chinner's talks are worth the time to watch if you're interested. You would be giving bad advice if you steered people one way or the other with regard to filesystems based on a seven-year old project (unless you've refreshed that system much more recently, of course).

> XFS has received some very serious performance improvements in the past couple of years to address indexing, large volumes of metadata, and so on, so that'd be one very relevant thing.

That's probably the difference right there. Thanks for pointing that out.

Your performance problem reminds me of this dentry cache performance failure https://sysdig.com/blog/container-isolation-gone-wrong/

Here is a link to benchmarks for the file systems. To bash XFS which is about as solid as one can get I would love to know why you would say that? http://www.phoronix.com/scan.php?page=article&item=linux-44-...

You have to tune the parameters at filesystem creation time if you care about small file performance. It was designed for large files.

What parameters? This guide[1] only mentions two things in relation to number of files. The first is inode count, for which performance is binary. The other is files in a single directory, and it says that the default setting is fine for a million. There's no explanation for the performance jacquesm describes.

[1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterp...

Reply on news.ycombinator.com