I'm at pretty much a loss as to why anyone who doesn't specifically need something that btrfs and btrfs
ALONE provides (writable snapshots from the top of my head although I'm not sure how that's different from a zfs clone.. One command?... I'm sure I read something cool about 'cp' being faster or whatever; is there anything else?) wouldn't go with ZFS instead.
Sure, you might need 5-10 lines of puppet or ansible to get it going in your linux of choice, but that's our job isn't it? And you're probably only using this on your storage boxes which are likely big pets instead of cattle (I use it on docker hosts too, but w/e)..
I've used ZFS in prod for 8 years at least (yay solaris, thumpers, fishworks etc) and it's never caused me any of the problems that things like VXFS, various sans, drbd etc have and I've run a bunch of pretty big infra on it (multi PBs).
Lets be realistic, you should really be picking your battles. Storage is almost always better to buy (read: cheaper/easier) than to build yourself unless you're one of the big boys (say, a cdn, some huge site with specific requirements or whoever.. Those folks probably have fs&kernel devs on staff).
As a responsible systems person you might tell your client just to give some money to emc or whoever as it'll be cheaper than having contractors build something where you need to care about filesystems, but if you absolutely must build it yourself, then go with the stuff that's mature, widely used and has all the nice features you might want from a FS (COW, Snapshots, compression, etc). Even then, probably your client won't have the staff to support your custom setup for more than about a year after you leave and they'll end up doing that anyway. Yes, I know some folks pulled it off; but I've not seen it yet...
As things stand, I cannot trust btrfs. I don't have a single reason to use a filesystem that has no users and has significant data loss issues in specific raid configs[0] when I have an alternative avail that works perfectly and is widely used and I have no idea why anyone else would either...
Losing data kills companies. You can't really fuck around with your software choices to do this stuff if you're a responsible person; save the fancy new tech for your apps not your infra....
There are, of course, gotchas with ZFS. You will get fragmentation on big pools over time which will hurt perf so you'll need to rebuild some storage nodes completely from time to time, but if you're building your own storage then this should be something you're capable of..
If you're custom building a few PB of redundant storage and your reason for not using zfs is you need to rebuild cluster nodes sometimes and you can't do that, then you shouldn't be be building storage on that scale.
BTRFS doesn't have that problem? Cool. I'd like to not have that problem too, but that doesn't mean that I'd use BTRFS just because I need to do those node rebuilds sometimes (which is good practice anyway isn't it?).
I'd maybe use in places I don't care about data safety, like as a storage driver for docker hosts (apps only) -- although I'd need a reason to bother trying...
Anyone?
0: https://btrfs.wiki.kernel.org/index.php/RAID56