Hacker News Reader

Comment by Teknoman117 | Hacker News Reader

Maybe I need to try ZFS again now that I have different drives. I used to run it on 6x 3TB WD Red drives (raid-z2) and could never get ZFS to perform anywhere near what I expected. Ended up going with LVM on mdraid (raid 6) on dm-integrity and got tremendously better performance for my "home nas / vm image host" machine.

View on HN

Measuring the performance of a monolithic RAID-Z array for a VM image workload is like measuring the 0-60 time of a semi tractor. That's not what that configuration is optimized for. Especially for a high IOPS workload, you should see much better performance with a pool of mirrors.

Your requirements sound similar to my home network. In my case how I orchestrated the spinning rust ended up being a lot less important than getting the random access parts of my workload on an NVMe SSD. Best $40 upgrade I ever made.

Yes you should look at that numbers: https://arstechnica.com/gadgets/2020/05/zfs-versus-raid-eigh...

Use Btrfs. Seriously. It's a tremendous fs. Read the caveats concerning raid56 and success.

My stack at the moment is 6x [10 TB WD Red Pros -> dm-integrity] -> mdadm raid6 -> dm-crypt -> LVM. One of my LVM volumes is btrfs. I only use LVM because I've had terrible performance for VM images stored in btrfs volumes. You can always use fallocate and disable CoW for the image but that prevents taking snapshots.

I'd if there was a zvol equivalent in btrfs.

Neither fallocate nor nodatacow will prevent taking snapshots. The result of writing to nodatacow shared extents is that the write must be COW, not an overwrite. Same as a reflink copy of a file on XFS. New "overwrites" are COW, subsequent overwrites are in fact overwrites.

We've phasing out using btrfs for RAID at work. It has been just a lot more straight-forward and easier to manage MD for the RAID-1 on our servers. We still use btrfs for container filesystems, subvolume snapshots are so handy. But that is btrfs on top of a MD RAID array.

Oh I'll get right on downloading that to my FreeBSD storage boxes, oh wait a minute.....

Reply on news.ycombinator.com