No, that's flat out not true.
I've seen that metric thrown around when talking about the "dedup" feature of ZFS, but honestly, don't use dedup unless you know what you're doing. It's way to easy for things to go wrong otherwise.
I've seen that metric thrown around when talking about the "dedup" feature of ZFS, but honestly, don't use dedup unless you know what you're doing. It's way to easy for things to go wrong otherwise.
Also, bup ("it backs things up!") efficiently dedups across an ssh connection (using bloom filters) Scales are differrent, but it might work for ZFS as well.
If you're short on RAM (sub 4GB), you might need to change some of the default settings to avoid problems, but RAM's fairly cheap nowadays, so unless you've got an old machine, it's not likely to be a problem :)
There's been talk in the developer community about ways to address the usability of dedup, but so far nothing has gone further than small prototypes.
https://github.com/zfsonlinux/zfs/commit/0dfc732416922e1dd59...
Aside from that, you are right that there has not been much done here. Making data deduplication more performant is a difficult task and so far, no one has proposed improvements beyond trivial changes.
I'm not sure how much benefit compared to the "real time" deduplication this approach can provide however, as the mapping table would still need to exist in memory, but I think there should be an increase to the write performance of non-duplicate data.
PS. My use case is that I have a few dozens of (linux-vserver) gentoo containers that obviously share many files and, unfortunately, trying to maintain a shared read-only mount of the core system doesn't seem to be practical/viable (as it does i.e. for FreeBSD jails, due to mostly the clear system seperation). The waste is not significant enough to really bother me, it would just be nice to avoid. The solutions that I am aware of are (not sure if I'm missing any, I'd be happy to be pointed to something else, if there is):
- integrated FS-level deduplication (ZoL, BTRFS)
- higher-level deduplication (lessfs and opendedup)
- hard-linking scripts (obviously at the file-level)