Hacker News Reader

Comment by kalleboo | Hacker News Reader

The major downside to ZFS-based systems like TrueNAS is that for a home or small business user you can't expand the storage with a new drive when you're running low on space. It's designed for data centers where you can afford to build a whole new array when you need more storage.

With Synology you go "oh, I'm down to 1 TB free, well there's this deal on a 10 TB drive, pop it in, now I have 11 TB free"

View on HN

This isn't really true: you can add new vdevs at any time: when you start running low, you just buy two drives and install them.

Right, 2 drives. Not "a new drive". Now you're buying twice as many disks as you would with a Synology and wasting 50% of your capacity on parity. And you better have set up your initial array with 2 drive vdevs as well or you're going to have a sub-optimal experience.

This is the attitude I see a lot in ZFS support forums. "I don't see the problem, just buy twice as many drives!"

> This is the attitude I see a lot in ZFS support forums. "I don't see the problem, just buy twice as many drives!"

This is incorrect on several levels.

You most certainly can create a vdev with a single drive in it and add it to the zfs pool. So go ahead, buy that single 10TB drive and add it to your pool.

That's not a wise thing to do though, so I don't understand why you'd want to. You'll have no redundancy at all, as soon as the drive dies everything is lost. Which pretty much completely defeats the point of having a NAS. So don't do that. But if you really want to, you can.

But doesn’t that come back to their point? With syno you pop in a new disk and it rebuilds the array with the new disk and you have more space and the same redundancy? Raid 5/6 whichever

With btrfs, you can add one or however many new devices you want to a storage pool, then rebalance to ensure redundancy across the whole pool. If the device you add is already btrfs formatted, its contents get added to the storage pool, rather than requiring a reformat.

It really surprises me that zfs apparently cannot do this.

The main reason I use btrfs is the flexibility. Subvolumes instead of partitions, and easy expandability. Storage should be dynamic, not static.

They have dRAID, but last I checked RAID 5/6 is basically asking for data loss with modern drive sizes.

> last I checked RAID 5/6 is basically asking for data loss with modern drive sizes

This is a debate I would love to see with people who have experience. Since I've seen individuals speak with authority on both sides.

I get that if you have a basic array of disks humming along with a big-ass ext4 partition, once one drive dies, the risk of the other drives being riddled with errors is huge.

But what if your array is both (1) using ZFS or BTRFS (with data checksumming) and (2) has scheduled full-disk data scrubs once a month or so? Wouldn't you catch the initial recoverable errors quick enough?

> Wouldn't you catch the initial recoverable errors quick enough?

Not always no.

I've had drives reporting failures for months that zfs scrub keeps fixing, tons of time to get a spare.

But drives also fail suddenly with no history of zfs or SMART errors.

> It really surprises me that zfs apparently cannot do this.

Likewise. I really want to like ZFS, but with the 'buy twice the drives or risk your data' approach as above really deters me as a home user.

ZFS has been working on developing raidz expansion for a while now at https://github.com/openzfs/zfs/pull/8853 but I feel that it's a one-man task with no support from the overall project due to that prevailing attitude.

BTRFS is becoming more appealing, even though it has rough edges around RAID write holes that really isn't a big deal, and reporting of free space. I can see my home storage array going to BTRFS in the near future.

> The main reason I use btrfs is the flexibility. I agree, and I as a small home user, I really like the RAID using different sized disks. E.g. running a raid 1 on three disks: 2TB+4TB+6TB. It also offers the possibility to increase the storage size over time when drives fail by replacing them with a larger disk.

> I don't understand why you'd want to

I want to add a single drive since I can't afford more than a single drive. But I still want to keep the data security of one or more parity drives. Synology lets me do that. ZFS doesn't.

On a Synology NAS (which just uses Linux mdraid underneath the hood so this part isn't exactly some proprietary magic) if you have an array with parity (the equivalent of raid-z/z2), you can add a drive, and it expands the array with that one drive, keeping the parity and recalculating it for the new configuration of drives.

So I can go from an array of 3 x 10 TB disks where one is parity (20 TB usable storage), and then just pop in one more disk and now I have an array with 4 x 10 TB disks (30 TB usable storage) with the same one-disk parity. I can lose any one disk, and lose no data.

ZFS can't do that, since it does't support modifying vdevs. So if I want to be able to add a single drive and expand my storage at any time while keeping the same level of redundancy, ZFS makes no sense.

Synology's configuration of mdraid+BTRFS makes way more sense than ZFS. Unfortunately they haven't contributed it to free software so nobody else can have it (specifically the part of passing through the parity data so that checksum errors in BTRFS can be fixed with mdraid knowledge). I would prefer to not have to rely on Synology's cost-cutting hardware and raft of probably not very secure software. But for the use case of me and the small businesses I support, ZFS has been a non-starter due to the costs.

> So I can go from an array of 3 x 10 TB disks where one is parity (20 TB usable storage), and then just pop in one more disk and now I have an array with 4 x 10 TB disks (30 TB usable storage) with the same one-disk parity. I can lose any one disk, and lose no data

Based on those numbers and https://www.synology.com/en-us/support/RAID_calculator I'm guessing you're using RAID-5?

RAID-5 is fragile. You can lose only one disk as you say, but the odds of succesful rebuild are not so great (assuming you have a NAS for data reliability in the first place).

https://www.digistor.com.au/the-latest/Whether-RAID-5-is-sti...

> expand my storage at any time while keeping the same level of redundancy

But you don't keep the same level of redundancy when adding a drive. The more drives you add in RAID-5, the lower your probability of a successful rebuild after the loss of one drive.

It was just an example with easy to reason about numbers. You could do the same thing with 2-disk redundancy.

> https://www.digistor.com.au/the-latest/Whether-RAID-5-is-sti...

I've seen a lot of articles and blog posts like this, but their numbers never seem to make sense. It says that reading through a 4-disk 8 TB array you only have a 15% chance of success. I have full-array BTRFS scrubbing scheduled monthly, according to this my array should have reported errors many times a year...

And of course, no matter what, no form of RAID/ZFS is a backup.

User of both QNAP devices and one of the iX systems devices here.

ZFS is sexy, but it requires planning and understanding and (as stated by another poster) adding storage in pairs of drives if you want to increase storage incrementally and maintain drive redundancy.

One of the perks of something like a QNAP or a Synology is the support for simply adding a single new drive to an existing RAID5 or RAID6 array, and having the storage box add it transparently while data is migrated to the new, larger RAID array. You pop in another 10TB drive in your RAID6 array and you increase the size of the array 10TB as you'd expect.

Or, if you've finally outgrown your 6-bay device which is full of 3TB drives, you can replace the existing drives with 12TB drives, then once they've all been replaced increase the size of the array to match the new drive sizes. This is done while the device is running and serving data - no downtime, though things may slow down as you would expect during migration operations.

From an end-user perspective this is a very different experience. Yes, FreeNAS/TrueNAS is cool, but I put a Synology at my dad's house.

The scenario you mention can easily be done with ZFS as well. I run a raidz1 and recently migrated from 3x4TB drives to 3x10TB. I bought one drive at a time and gradually expanded the pool. For each new drive I added I simply had to resilver the pool and I was done.

You most certainly can add drives to a zfs pool to expand it.

I've been running zfs on my file servers for ~17 years, have expanded the pool many times. In all that time I've only built a new machine once. Currently still running on my 2009 file server build. I've swapped and added drives to it over the years though.

> You most certainly can add drives to a zfs pool to expand it.

You can't replace drives with bigger ones and expand the pool. This is important, if you have 4/5/6/8 bay chasis and exactly the same amount of drives in the pool.

>You can't replace drives with bigger ones and expand the pool.

Yes you can. That's exactly what the 'autoexpand' property is for. It's odd how this kind of thing floats around on the internet.

Just because there's a property doesn't mean that it works. It floats around on the internet, because that's the experience people have.

It works just fine. I've upgraded vdevs this way several times, and it worked without a hitch on every occasion.

It didn't for me; in the end, I backed up the files, scraped the machine and put the drives into Synology box.

So take it as an piece of the puzzle why is Synology more popular.

You can, although you need to replace all the drives in the pool. You can swap one, wait for resilver (or for a month, if you're in a budget), do the next one... And once you replaced every drive, you do like this:

https://www.ateamsystems.com/tech-blog/expand-zfs-to-use-lar...

Few years ago, I needed exactly that, and it didn't work.

I guess I'm not alone: https://www.google.com/search?q=zfs+autoexpand+not+working

You can add drives to a ZFS pool, but you need to either replace or add them in massive chunks (or smaller chunks if you're happy buying 2x as many disks as you actually need).

If I want one disk redundancy.

Today I can afford 2 10 TB disks.

Next year I need more than 10 TB capacity and I can afford one more disk.

Two years from now I need another 10 TB capacity and I can afford one more disk.

How can I perform this migration with ZFS? Going from 10 TB - 20 TB - 30 TB of capacity, adding one disk at a time, without losing redundancy.

Or say next year and two years from now 12 TB drives are cheaper. So with (10TB+10TB) + (12TB) + (12TB), Synology will give me 32 TB of usable space and I will have one drive redundancy throughout the whole time.

Honestly curious, this is a real-life situation that me and several of my friends have done with Synology NAS. For this use case, I would love to use cheaper and more performant used hardware, and not have to rely on proprietary software that phones home. ZFS requires upgrading your disks all at once, unRAID has single-disk performance, straight-up Linux BTRFS is "unstable".

> Honestly curious, this is a real-life situation that me and several of my friends have done with Synology NAS.

I guess I don't understand why optimize for the cost of a single drive, above all criteria?

Between this and the other comments, you've mentioned that Synology is over-priced, lower quality, lower performance, proprietary and phones home. Are you really better off vs. building a higher-quality more performant lower-cost ZFS server that's fully open source and has better reliability?

If Synology is higher cost, maybe take that difference in price to buy an extra drive or two?

To me a NAS is all about reliability.

> and I will have one drive redundancy throughout the whole time

Mentioned in the other comment, but that's not a good way of looking at it. What matters is the probability of loss of data while rebuilding the data after one drive has died. The more drives you have in that set, the larger the probability of loss. Your risk is increasing with every drive you add.

Synology hardware is overpriced for what it is, but in the home-sized NAS segment, it's still way cheaper than buying drives.

I simply can't afford to buy a whole array upfront. I can just afford to expand it every other year or whatever.

If you're comfortable with the large and ever increasing risk of loss (by adding drives without adding redundancy) then Synology is probably indeed a better match for your use case then ZFS.

I don't really understand why pay for dedicated NAS hardware if reliability isn't priority #1, but that's me.

Personally, for stuff that I care about but not quite that much, I just keep on the SSD on my laptop. It'll very probably be fine but there is risk of loss (same as Synology).

For the things I care deeply about, they go on the ZFS server with tons of redundancy, snapshots and backups. I'd never trust the truly precious data to anything other than ZFS.

Reply on news.ycombinator.com