XFS, ext and per-inode mutexes | Hacker News Reader

XFS, ext and per-inode mutexes

facebook.com

43 pointsGarbage17 comments

I'm starting at Facebook in about a week and a half and a big reason I chose Facebook over Google as an infrastructure guy was (somewhat ironically) their openness (in infrastructure/engineering practices).

I much prefer the quick Facebook note to the occasional well-edited white paper.

Good luck to you!!

Mark didn't mention this in his article, but writes to the inode are only serialized in jfs/ext(2|3|4) if you are using direct I/O (i.e O_DIRECT) which for performance reasons MySQL ibdata files often are. This is why this problem does not often show up on filesystem benchmarks.

I like both XFS and JFS filesystems. I have used both on and off with Arch Linux and I've never had a bit of problem with either. Some people don't fancy them because they are in maintenance mode now but, frankly, I would rather use a finished and maintained filesystem over something that is in constant development (unless the new filesystem in development provides something truly unique).

I think the one thing I miss from BeOS (my all-time favorite OS along with Nextstep) was BeFS. It was nice to be able to create arbitrary file metadata and then have it indexed and searchable in real time. Does any Linux FS have this capability? I believe that XFS supports extended metadata but from what understand it is not something one can search on.

You can't inherently search on it (i.e the filesystem doesn't manage its own index) but it would be trivial (with something like inotify or its successors) to write an app to monitor the filesystem for metadata changes and index them itself. This is how Spotlight works on OS X, with HFS+'s arbitrary metadata - the spotlight daemon watches for FS events and updates its index when a change occurs.

I strongly suspect that this is actually what was happening in BeOS as well, at least on some level. It would be surprising to me if the filesystem itself was also what maintained the index; it seems like this would reduce write throughput if it had to update the indexes all the time as well.

I think GNOME Tracker and Zeitgeist are going in this direction.

I wouldn't say XFS is in maintenance mode, it received important enhancements in the recent past, and significant performance gains.

Does anybody know the ext4 behaviour?

Yes, ext4 uses inode mutexes by default like ext3. They're needed if you want to use a journal, for example. Open http://www.kernel.org/doc/Documentation/filesystems/ext4.txt and look for dioread_lock for more information.

Also, does anyone know what does btrfs do for writes? Its .txt is considerably shorter than ext4's unfortunately.

(I could only find "Btree lock contention: The btree locks, especially on the root block can be very hot. We need to improve this, especially in read mostly workloads." in the unclaimed projects which seems a bit related)

I used XFS for years, until I got tired of it semi-randomly overriding my files with zeros and then found out that apparently this was 'by design', after that I stick with ext3/4 that might not be as fast in some corner cases, but which are just as fast or more with my usage patterns, and which so far has been good at not corrupting any of my data.

semi-randomly overriding my files with zeros and then found out that apparently this was 'by design

Can you elaborate on this?

I'm using XFS in a few fairly large deployments and so far without any problems. Should I worry?

I believe not, unless you are in the habit of losing power without getting a chance to unmount your filesystems cleanly.

Someone please correct me if I'm wrong, but my understanding is that it's something approximately like this:

In the event of an error and journal replay, the filesystem metadata can be recovered, but not necessarily the data itself. ext3 tries to keep your data intact, or at least as much of it as it can piece back together. XFS (and I think JFS), if they don't know for sure that all the data is intact, just zero the file. This is by design, on both sides.

So in normal usage everything is fine; it's just a difference in error handling.

No, because XFS doesn't eat your data if you're using decent hardware with decent applications. See

http://flamingspork.com/talks/2007/06/eat_my_data.odp

It's probably not the best choice for a basic desktop without UPS, though, or a laptop.

XFS Zeroes the block instead of keeping the random/sensitive data that was already on the disk if the metadata necessary for that write was not commited before the failure. Ext leaves the data as is.

If you want full protection against those situations instead of praying that it's still the right data or that it doesn't happen at all, you need a copy-on-write FS, or enable data=journal on ext3, at a massive performance cost.

Anyone know if btrfs would be more like XFS or ext3/4 in this regard?

Reply on news.ycombinator.com