Hacker News Reader

Comment by throw0101a | Hacker News Reader

> So it's basically a matter of fixing various software to work nicely with btrfs as well as any similar filesystem. I do wonder why it isn't more supported by upstream software (postgres, VM things, etc).

The concept of an application 'supporting a (specific) file system' sounds slightly ridiculous to me.

Why is Btrfs so special that apps should be rewritten for it? The always-compared ZFS seems to work just fine without alterations being necessary, so what make Btrfs worthy of extra considerations?

At most perhaps copy-on-write (COW) file systems as a general class could perhaps be considered, as that may help with things like shingled drives and SSDs (i.e., zoned drives may be aligned with COW FSes).

View on HN

Historically programs are "rewritten" (really, adapted) all the time to support changes to the underlying system. E.g., read-ahead, mmap() support, fadvise() support, file locking, different file locking semantics and failure modes in network filesystems, sparse files, hashed directory entries, asynchronous I/O, changes to underlying media characteristics like SSD seek times, etc.

> The concept of an application 'supporting a (specific) file system' sounds slightly ridiculous to me.

This concept is actually way less ridiculous than you might think, for any application which needs any guarantees about data durability, locking, etc (which includes everything from the obvious ones like postgres to things like Dropbox). I found https://danluu.com/deconstruct-files/ a fascinating read diving into this.

Because Fedora, among other distributions, will not include ZFS for licensing reasons. (Leaving aside any other discussion of the relative merits of ZFS and Btrfs.)

ZFS has a good share, if not 100%, of the same problems as Btrfs, and in the end of the day it's always matter of leaving files/directories/volumes without CoW, which is hardly a software rewrite as the grandparent insinuated, and more of a packaging issue.

>ZFS has a good share, if not 100%

ZFS has no Raid5 write-hole...no traditional Raid problems at all (z1/2/3)

>leaving files/directories/volumes without CoW

No...just NO, just for a special Cases like a Database.

> ZFS has no Raid5 write-hole...no traditional Raid problems at all (z1/2/3)

The thread was about where CoW is problematic, both ZFS and Btrfs are identically problematic in this sense.

> No...just NO, just for a special Cases like a Database.

Like... for what we were talking about.

To me, it doesn't sound any more ridiculous that applications having specialized support on different architectures. Most applications get away without it, but when performance really matters and general-purpose abstractions aren't sufficient, having special cases seems okay—especially when there's only a small number of architectures (or filesystems) to handle.

This can go too far—I still remember web development a decade ago—and I would prefer to have perfect abstractions that never need piercing, but absent that specialized logic seems _fine_.

> The concept of an application 'supporting a (specific) file system' sounds slightly ridiculous to me.

Clearly you weren’t around in the days when NFS was a thing.

Wait, NFS isn't a thing anymore?

> Clearly you weren’t around in the days when NFS was a thing.

I was and am. I currently help admin an HPC environment with several petabytes of NFS storage.

And the main problems I've experienced with NFS have generally been with Linux locking. I've had a lot less drama with Solaris and BSD NFS (though I did uncover two Solaris NFS server bugs over the years).

Whenever I have NFS issues I generally start with the assumption that there's a bug in the NFS client code.

If you think applications work on NFS by just expecting general Unix filesystem semantics you’re in for a surprise.

The fact you’re not having trouble just means either the application writers put in the effort to properly support it, or your use case is simple enough not to run into problems.

> either the application writers put in the effort to properly support it

Given the hiccups that we've had with MySQL and locking, I'm not sure that's the case. (Less so with Postgres.)

I have only a moderate amount of knowledge of the applications that the dozen research groups use on the cluster, I just make sure the bits flow and haven't heard too many complaints over the years.

Clearly packages like MySQL and PostGreSQL made sure they work on NFS or they would very clearly tell you not to use it because they’d know it’s a recipe for disaster. The issues I am referring to typically are with things like homegrown shell scripts or research software.

Reply on news.ycombinator.com