Hacker News Reader

Comment by carlob | Hacker News Reader

> Many remain skeptical of deduplication, which hogs expensive RAM in the best-case scenario. And I do mean expensive: Pretty much every ZFS FAQ flatly declares that ECC RAM is a must-have and 8 GB is the bare minimum. In my own experience with FreeNAS, 32 GB is a nice amount for an active small ZFS server, and this costs $200-$300 even at today’s prices.

I use nas4free with much less ram…

View on HN

The massive amounts of RAM recommendation is if you need to do deduplication. Are you doing that? If not then you don't need a lot of RAM.

Between the low cost of storage, and alternative solutions for deduplicating data I personally don't use the built-in deduplication functionality of ZFS for my zpools. Might come down to what sorts of data you are storing, though.

Yup. This. Now that I have 32 GB of RAM in my FreeNAS box I decided I really didn't need dedupe after all. I just don't have that much duplicated data, and I've got 60 TB of HDDs in the box. So I use the RAM for VMs instead!

Yeah dedup is mostly useful for massively multiusers setups, say mail, file sharing, etc etc. For small setups you can do a `fdupes` pass once a day and fix it yourself I suppose.

I remember that time I turned on dedup on our 45drive box with 32GB of ram and 140TB~ of data..

...and nothing happened because it does not retroactively dedup existing data?

Over time it would become horrendously slow, I agree, since you have too little RAM by a factor of at least 8.

And this is why ixSystems disables access to the dedup switch on their TrueNAS systems without contacting support to enable it.

I've got 30TB of small files stored on our TrueNAS system at work, there's no benefit to enabling it but if someone decided to toggle it we'd quickly learn there isn't enough RAM in the world to handle billions of 1-16KB files....

Reply on news.ycombinator.com