Hacker News Reader

Comment by gsmethells | Hacker News Reader

ZFS forces you to choose a drive size a priori and live with it for the life of the system. Thanks but no thanks. I'll stick with OpenStack Swift where I can leverage new larger disks and have my uncoupled highly available storage to boot.

View on HN

This is not true. Read up on mirrored vdevs. http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-...

50% storage efficiency and 87.5% survival is not an option for us. We store medical images by the petabyte and drives fail much too fast at scale.

Especially with the long resilver times necessary to achieve a good ROI from the investment.

It's great to hear how you are using Swift. I work on that project; feel free to reach out if you have any questions. Contact info is in my profile.

That is not correct: one can easily increase the size of the pool by adding larger devices. As soon as the last device is replaced, the pool instantaneously possesses the upgraded capacity. No complicated grow commands, no filesystem expansions. It JustWorks(SM).

  zpool set autoreplace=on pool_name
  zpool set autoexpand=on pool_name

properties must be set on the pool for this to work, either before or after the pool upgrade. Note that older versions of zpool(1M), like for example zpool version 15, do not have the autoexpand property.

What cannot be done is a reduction in pool's capacity, but that does not come into these tales.

There is no "apriori size determination", as zpool(1M) uses the entire disks or logical units, and ZFS filesystems use only as much capacity as they require, and no more, leading to extremely efficient disk space utilization.

That is a lot of work compared to just adding a disk and rebalancing. No need to replace every disk in a vdev just to increase disk utilization to the entire disk when using Swift. No fear of losing a pool when losing a vdev.

"Increase disk utilization when using Swift"? What you wrote makes no sense.

There is never any fear of losing a pool when losing a vdev, as ZFS will let one simulate the entire thing by letting one use files as disks:

  % mkfile -v 128m /var/tmp/d0
  /var/tmp/d0 134217728 bytes
  % mkfile -v 128m /var/tmp/d1
  /var/tmp/d1 134217728 bytes
  % sudo zpool create testpool0 mirror /var/tmp/d0 /var/tmp/d1
  Password:
  % zpool status
  pool: testpool0
  state: ONLINE
  scan: none requested
  config:

	NAME             STATE     READ WRITE CKSUM
	testpool0        ONLINE       0     0     0
	  mirror-0       ONLINE       0     0     0
	    /var/tmp/d0  ONLINE       0     0     0
	    /var/tmp/d1  ONLINE       0     0     0

  errors: No known data errors
  % zfs list
  NAME        USED  AVAIL  REFER  MOUNTPOINT
  testpool0   988K  79.0M   952K  /Volumes/testpool0
  % sudo zpool set autoexpand=on testpool0
  % mkfile -v 192m /var/tmp/d2 
  /var/tmp/d2 201326592 bytes
  % mkfile -v 192m /var/tmp/d3 
  /var/tmp/d3 201326592 bytes
  % sudo zpool replace testpool0 /var/tmp/d0 /var/tmp/d2
  % sudo zpool replace testpool0 /var/tmp/d1 /var/tmp/d3
  % zfs list
  NAME        USED  AVAIL  REFER  MOUNTPOINT
  testpool0  1008K   143M   952K  /Volumes/testpool0
  % sudo zpool destroy testpool0
  Running process: '/usr/sbin/diskutil' 'unmount' '/Volumes/testpool0' 
  Unmount successful for /Volumes/testpool0
  % rm /var/tmp/d[0-9]

Did you not read the other guys link above? http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-...

It says "Fault tolerance / degraded performance

Be careful here. Keep in mind that if any single vdev fails, the entire pool fails with it. There is no fault tolerance at the pool level, only at the individual vdev level!"

I've been architecting and implementing highly fault tolerant, large scale storage solutions with ZFS since one week after it came out at end of June 2006, but thank you, I'll be extra sure to keep that in mind.

Please do not assume that everyone here comes from GNU/Linux and knows next to nothing or half-true anecdotes about (Open)ZFS. For example, I learned what and how in ZFS directly from the ZFS development team at Sun Microsystems: Roch Burbonnais, Adam Leventhal, Eric Shrock, and Jeff Bonwick.

I didn't say "increase disk utilization with Swift", I said ZFS does not use the whole disk when you add a new large disk unless you replace the entire vdev worth of disks. Swift treats each disk individually and uses weighting to deal with heterogeneous disk sizes which is much cleaner.

Replacement of entire disks to grow the pool is simply a different way of adding capacity, because ZFS is based on different paradigms than what one is used to with run of the mill implementations; however, in practice, it's a much simpler and therefore much more reliable way to extend capacity. The units are simply entire larger disks.

Swift is also how many orders of magnitude more complex?

Reply on news.ycombinator.com