Hacker News Reader

Comment by shmerl | Hacker News Reader

So it's a layer under the filesystem and should work with any filesystem in theory? So would XFS + VDO be better than let's say using compression in BTRFS?

View on HN

Yes, VDO is transparent for the layers above, like filesystems. XFS+VDO might be preferable to BTRFS, if it has the features you need. - LVM could be used for snapshots, but the snapshots from BTRFS might be preferable in corner cases where many snapshots are done (snapshots in XFS are being worked on AFAIK, these could then be comparable) - XFS+VDO combination might be more reliable, it's a completely supported combination on RHEL, whereas BTRFS is just techpreview - only missing feature coming to mind is checksums over data blocks, XFS is only providing checksums over meta data

How do filesystems cope with the apparent capacity of the disk changing? If you put a lot of data on that compresses really well, suddenly your disk will appear to have become larger.

The capacity is not changing 'rapidly', instead the whole VDO device is thin provisioned. Let's assume a 1TB harddisk, then when creating a VDO device ontop, you can have that directly appear as 3TB, so 'thin provisioned'.

The filesystem ontop always sees 3TB, unless you explicitly modify the VDO device. Of course, you have to monitor the VDO status tightly: if you happen to store data on the filesystem which is absolutely unique, has no zeros and is uncompressable, then dedup/compression can not do anything. The VDO device can only consume up to ~1TB of such data. Your monitoring should detect the low space on the VDO backend before, and you should then either stop writing or extend the backend device.

What happens if you go over the limit? Does VDO reject writes? Or is your filesystem lost.

With the current code, your data becomes unmountable in that case. There is a workaround for getting the data accessable: leave a bit of the underlying blockdevice unused, so when creating VDO ontop give it not the full blockdevice. Then, in case the filesystem ontop completely fills up, the last part of the blockdevice can be used to grow the VDO device, and the filesystem can be mounted.

Should be tried out before relying on it. The current behaviour seems to be considered a bug, https://bugzilla.redhat.com/show_bug.cgi?id=1519377 has details.

I’ve been wanting to try Cassandra on top of ZFS to see if the file system compression worked better than Cassandra’s internal impl. This would be much easier to try (and to sell to my manager) with the Redhat support.

I'm assuming so, because Redhat is getting rid of btrfs. That would explain why they announced the two in the same release notes.

Reply on news.ycombinator.com