Hacker News Reader

Comment by miohtama | Hacker News Reader

Does XFS offer benefits or tradeoffs of ZFS/ext4/others today?

I have been using it with Linux since early 2000. But I am not sure how the file-system development has been lately e.g. with the SSD revolution.

View on HN

If you look at database benchmarks on TPC.org, you will find that many of them are on XFS (as a consequence of using RHEL/CentOS), which was designed with database performance in mind.

It used to be necessary for Oracle to use raw partitions to get the very best database performance. I haven't heard of anyone doing that for many years.

"The default implementation in most kernels is simply a file-global lock placed at the in-memory inode, making sure there can be only one writer per inode. Implementers of databases hate that because it limits the write concurrency on any single file to One. This is, for example, why Oracle recommends that you make tablespaces from multiple files, each no larger than one GB.

"XFS, in O_DIRECT mode, removes this lock and allows atomic, concurrent writes, making database people very happy."

Developers at Oracle have posted a number of blog posts on their code changes for Linux XFS.

https://blogs.oracle.com/linux/post/introduction-to-xfs-tran...

https://blogs.oracle.com/linux/post/xfs-online-filesystem-ch...

https://blogs.oracle.com/linux/post/xfs-2019-development-ret...

https://blogs.oracle.com/linux/post/xfs-whats-new-and-whats-...

There can be a strong bias against Oracle at HN (and I understand why) but credit must be given where it's due.

ZFS is also from Oracle, right? Seems like a dominating force in the FS industry if one can call it a such.

I mean, in that it came out of Sun, kind of I guess.

Btrfs is the filesystem for which they are most responsible.

They explicitly do not support its use for their database.

"[Btrfs] was initially designed at Oracle Corporation in 2007 for use in Linux, and since November 2013, the file system's on-disk format has been declared stable in the Linux kernel."

https://en.m.wikipedia.org/wiki/Btrfs

They own the rights to ZFS by way of their purchase of Sun Microsystems.

Oracle is a power player in the realm of Linux filesystems, no doubt.

Linux uses OpenZFS, which is a fork of OpenSolaris' ZFS; they haven't shared any code since 2010 AFAIK.

ZFS is Sun.

BTTFS is Oracle.

And neither have anything to do with a database, IMHO.

Oracle DBAs used to recommend using raw block devices to back the dB, a long long time ago.

I never did get one to explain why. I suspect it was received wisdom depending on VFS models which cached disk io and an effect on the DB engine. Almost certainly overtaken by both disk controller logic, file systems cache, striping and RAID and increasing memory sizes.

There are good pages on the Web discussing optimising ZFS for postgres tuned to write disk block aligned data through the zfs L2ARC

> Almost certainly overtaken by both disk controller logic, file systems cache, striping and RAID and increasing memory sizes.

In practice this might be true if the defaults settings of each layer happen to align sympathetically and/or you know what you’re doing and how to tune them all, but from first principles it stands to reason that the extra abstraction layer of a file system isn’t providing the database storage engine with anything useful?

Same argument for unikernels, distroless containers, LiquidVM (https://docs.oracle.com/cd/E13223_01/wls-ve/docs92-v11/confi...) etc…

Yes. It's still a valid proposition that raw disk is fastest if you map cleanly to the block abstraction the integrated controller offers, if only because of avoidance of indirect call stacks to carry data into the device.

But that said: the DBA in question constructed a catenated drive, not a stripe, and were slightly dismayed when we found out it was working at the speed of one disk interface and using only the front pack, not all of them in some kind of parallel or interleaved manner. Not a very smart DBA it turned out.

The reason for using raw/block devices back then (or Veritas VxFS that created additional raw devices to access VxFS files in a manner that bypassed regular OS filesystem I/O codepath & restrictions for reads & writes) was concurrent I/O.

In a busy database there are lots of processes reading a datafile and multiple processes writing (like DBWR checkpointing dirty blocks from buffer cache, but also some others), so this caused unnecessary file level contention in the OS kernel. Note that Oracle has I/O concurrency control built in, you can't have two processes accidentally writing the same block to the storage at the same time or a reader reading a "torn block" due to an ongoing write (the serialization happens at buffer lock/pin level in Oracle code).

So you wanted to have concurrent I/O and you either had to use raw devices or Veritas VxFS back then. Later on concurrent I/O became available for some other filesystems too, like Solaris UFS having the "cio" mount option, etc.

Note that Oracle still uses "raw" block devices for high-performance configurations. It's called Oracle Automatic Storage Management (ASM), basically you give it a bunch of block devices without any partitions or file systems on them and ASM does its own "extent" placement, striping, mirroring, snapshots, etc. And Oracle's Exadata scale-out appliance also uses block devices without any filesystems directly, the DB nodes talk to any storage nodes over InfiniBand/RDMA (older hardware) or RDMA over Ethernet/RoCE.

I think you meant ZFS, not zsh.

I use both XFS and ZFS. XFS in any situation where it is a single drive (ie, my laptop), ZFS in any situation where I'm doing actual fault tolerant high performance storage stuff.

I don't see where ext family really matters in modern storage. If I decide to not use XFS anymore, there are flash-dedicated filesystems that will beat ext in performance and reliability on SSDs that better fit the future.

Fixed, thank you!

ZFS is actually ideal for single drives because even if you can't do recovery, knowing a file has corrupted is extremely valuable.

set copies=2 or 3 and it can recover corrupted data on that filesystem at the expense of storage

SSDs are very likely to put all those copies into a single physical block underneath. Since ZFS makes backups easy, better stick to copies=1 and do backups often.

Wrong. ZFS' copies are made across several different devices.

What do you mean? The discussion in this particular sub-thread is about running ZFS on a single drive (think laptop). Does it have some kind of mechanism to send the write for a copy "later enough" that it will likely end on a different physical block?

I meant "primarily this copies mechanism is targeted towards multiple devices setup".

With a single SSD it's indeed prone to the caveat which was pointed out; even if not due to being mapped to same storage "area" but also because SSDs often fail completely.

Also makes sense to note that __when__ narrowed to a single-disk setup ZFS' can be interchanged with Btrfs; almost same set of features but lesser overhead and complexity.

Wrong. ZFS' copies are made across several different devices.

(2nd copy)

If you are using ZFS on a single drive, they reside on the same drive by necessity. Which was the case the parent posters were talking about.

I shouldn't think that's true. Why wouldn't the firmware be splitting writes across different physical chips for performance and wear leveling reasons?

> there are flash-dedicated filesystems that will beat ext in performance and reliability

If you mean f2fs, it really won't. Be careful with it.

https://www.usenix.org/system/files/atc19-jaffer.pdf

The paper is old at this point, but I found a fresh PhD thesis from the same author from 2023, and it seems to re-confirm these findings.

That sucks. I know a lot of Android users that prefer f2fs over ext* due to reliability and performance concerns, and seem to at least have some concrete reason to believe so.

Single drive ZFS is also a great use case for ZFS because:

- full data integrity check (not only metadata like with XFS)

- transparent compression (LZ/ZSTD/...)

- ZFS Boot Environments - makes your system bulletproof against changes

- easy move ZFS Boot Environments between systems with zfs send|zfs recv commands

None of that is possible with XFS.

I'm too lazy to summarize this beyond saying both ext4 and XFS have their advantages and disadvantages, and both are still under active development. ZFS is a funny and problematic one that does lots of cool tricks but isn't properly free and gobbles resources.

Check this: https://linuxhint.com/xfs-vs-ext4-brief-comparison/

ZFS (or rather, OpenZFS) is absolutely Free Software and Open Source under any definition. Richard Stallman will tell you the same thing.

It just has an unfortunate license that's not GPL-compatible and this causes problems with GPL kernels (e.g. Linux), which is a very different thing than "not being properly free".

Context:

https://en.wikipedia.org/wiki/Common_Development_and_Distrib...

However, the wiki article also notes that Ubuntu includes it by default as a binary module. That seems like a sensible thing to do from a legal perspective, since it's generally accepted that binary modules are not "linked to" the kernel in such a way as to trigger GPL provisions relating to such linkage.

"Don't use ZFS. It's that simple. It was always more of a buzzword than anything else, I feel, and the licensing issues just make it a non-starter for me." -Linus Torvalds

https://www.realworldtech.com/forum/?threadid=189711&curpost...

Even Linus gets it wrong from time to time. And here he is absolutely wrong.

ZFS is absolutely Free Software, it's just that its license isn't GPLv2 compatible. (Bearing in mind that by the same stroke, GPLv3 isn't GPLv2 compatible.)

Reply on news.ycombinator.com