Hacker News Reader

Comment by jacquesm | Hacker News Reader

I have one machine running XFS but if that one is representative then I won't be installing XFS anywhere else and would happily discourage others from using it. It is terribly slow when doing some fairly common operations when you have a large number of small files.

View on HN

You're joking, surely.

XFS has outperformed EXT4 in almost all "high" use-cases in my experience and testing: Large files (500GB~) or many small files (128k files * 2,400,000 or so). EXT4 under those loads is comically bad.

BTRFS is also terrible at this, only XFS and ZFS are good at handling it.

On the other hand on database workloads, for example PostgreSQL, XFS and EXT4 are about equal these days. ZFS (at least on Linux) and Btrfs are both clearly slower on those workloads.

Here is one benchmark, but I have seen plenty of similar benchmark results for PostgreSQL showing the same thing: https://blog.pgaddict.com/posts/postgresql-performance-on-ex...

There's a fairly simple reason for that which is that ZFS (and btrfs to some extent) are almost literally "ACID" databases. They do alot of the same double-writes and other safe behaviour the database is also doing. Those have a penalty and you're doubled up.

There are various guides around for tuning ZFS and database servers to try reduce that duplication, for example you can disable the InnoDB double write buffer because ZFS guarantees you don't need it. You also need to tune recordsize to match the database page size so that you don't accidentally create large multi page blocks.

I partially disagree with the claim that ZFS is slower than ext4/xfs. It is, but only as long as you don't use ext4/xfs on top of LVM to get similar snapshot capabilities etc. Then ZFS starts to win.

So if you only need a plain filesystem, ext4/xfs are great and you will get better performance.

If you need/want snapshots, e.g. to do backups that way, it makes sense to look at ZFS.

I wish I was. It's pathetic. Creating a new directory entry on an idle machine with plenty of CPU and memory takes seconds, ditto deletions.

Also: I love how that comment sits at -4, as if downvoting it will somehow discredit the data point.

> I have one machine running XFS but if that one is representative

> Creating a new directory entry on an idle machine with plenty of CPU and memory takes seconds, ditto deletions.

I think your answer lies in your premise then. It's not representative.

10's millions of files should have been the ideal use case for XFS, that's why I installed it in the first place. This was for the 'reocities.com' project and by the time I realized what the problem was most of the import had already been done so I let it run to completion but it makes updating the project a real PITA.

Sure, but the issue could be configuration, drive, interface, etc. It's impossible to speculate in, but what we know is you have trouble with one machine, and it's the only one that has used XFS. It's unfortunate, but likely a coincidence, or at least unrelated to XFS at its core.

I've been using XFS for 10 years without the issues you seem to be having.

There's so much that can go wrong setting up a Linux server that it's impossible to give much advice with something like this.

I guess the general stuff is: the easy default partitioning setup you get from a Linux distro is total bs, you need more RAM than you think you do, the way you're serving files or accessing the system (NFS!) has plenty of ways to screw things up as well, and tens or hundreds of millions of files is not any filesystem's ideal use case. The classic IRIX workload would be guaranteed-rate streaming of large media files, and the Linux port of the filesystem obviously inherited a lot of that system's traits (without the GRIO).

XFS has received some very serious performance improvements in the past couple of years to address indexing, large volumes of metadata, and so on, so that'd be one very relevant thing. Dave Chinner's talks are worth the time to watch if you're interested. You would be giving bad advice if you steered people one way or the other with regard to filesystems based on a seven-year old project (unless you've refreshed that system much more recently, of course).

> XFS has received some very serious performance improvements in the past couple of years to address indexing, large volumes of metadata, and so on, so that'd be one very relevant thing.

That's probably the difference right there. Thanks for pointing that out.

Your performance problem reminds me of this dentry cache performance failure https://sysdig.com/blog/container-isolation-gone-wrong/

You have to tune the parameters at filesystem creation time if you care about small file performance. It was designed for large files.

What parameters? This guide[1] only mentions two things in relation to number of files. The first is inode count, for which performance is binary. The other is files in a single directory, and it says that the default setting is fine for a million. There's no explanation for the performance jacquesm describes.

[1] https://access.redhat.com/documentation/en-US/Red_Hat_Enterp...

Here is a link to benchmarks for the file systems. To bash XFS which is about as solid as one can get I would love to know why you would say that? http://www.phoronix.com/scan.php?page=article&item=linux-44-...

Reply on news.ycombinator.com