Why did Apple drop ZFS? | Hacker News Reader

Why did Apple drop ZFS?

storagemojo.com

27 pointschaostheory17 comments

Perhaps the reason is that ZFS is actually not a big win for Apple's core product market(1): laptops and single user workstations (even that is grudging). From the original article:

Manage storage, not disks. You can put all your disks in a pool and specify the redundancy level. ZFS takes care of the rest.

Essentially all of Apple's products are single disk. There is virtually no win there.

No more silent data corruption.

The checksumming is a nice feature and desperately needed as disk capacity continues to explode but bit errors remain the same, but in practice probably the biggest risk is bugs in FS code:

http://www.techcrunch.com/2008/01/15/joyent-suffers-major-do...

Easy snapshots.

This is slick and gives the UNIX guy in me happy dreams but Time Machine really stole the thunder on this for regular users. It's just easier to explain to an end user "go to your backup and drag the files back" instead of the way that snapshots work in ZFS.

High performance software RAID built-in.

Again, one disk devices can't benefit from this

Transparent compression on the fly.

I challenge anyone to come up with a big win for this for the common Mac case. Most common or data-intensive file formats are compressed because it's just easier for any computer to deal with.

(1) Considering Apple's server products, xSAN is actually very slick, and in a lot of cases the flexibility of SAN storage outweighs the benefits you would get from ZFS's multi-device integration. The JBOD model demonstrated by Thumper + ZFS can be incredibly inexpensive but doesn't align practically with many workloads, including (I would expect) the video processing business, which is probably Apple's biggest server market.

> Essentially all of Apple's products are single disk. There is virtually no win there.

You can have multiple copies even inside a single disk. And this is useful when you use cheap disks.

> The checksumming is a nice feature and desperately needed as disk capacity continues to explode but bit errors remain the same, but in practice probably the biggest risk is bugs in FS code:

This is not true. At some point, a heavily used disk will start failing. It is really nice to be protected against that - and very important to know when it starts happening.

I talk from experience - I had a zpool scrub detect silent file corruption once. Curiously, it never happened again with that disk. But I keep using that disk (of course, with frequent backups) - and I would never do that (continue using a disk that failed) with any other Filesystem.

Essentially all of Apple's products are single disk. There is virtually no win there.

Well, if you're data-paranoid, like me, you can set your ZFS filesystem to keep 2 copies of every file on different parts of the disk. Then if your single disk has a sector or two go kaput, ZFS will notice the bad CRC on read and get the other copy.

I've also heard of other improvements for single-disk, and SSD specifically, but alas, I can't find citations right now.

Time Machine really stole the thunder on this for regular users.

Time Machine's "snapshots" suck. Entirely new file every time a bit changes? Really? My 500 gig external disk just filled up. And it's only backing up a 250 GB drive. And I only started using Time Machine a few months ago. Virtual machines & TrueCrypt volumes that change often are a royal pain with TM. ZFS, on the other hand, can store 4 months worth of backups of a 1TB RAID array on my fileserver in way south of a gig.

Also, ZFS send and ZFS recv are awesome ways to stream a filesystem snapshot over a network to, say, a Time Capsule. If you were so inclined.

Actually, it's worse than that. Time machine's "snapshots" aren't snapshots - they don't represent a point in time at all. A time machine backup can complete and nevertheless be unrestorable. On filesystems that support snapshots - including Windows' - you can actually do a reliable backup, but not on OSX.

Time Machine's "snapshots" suck. Entirely new file every time a bit changes? Really? My 500 gig external disk just filled up.

Yeah, this has been far less useful than I imagined it would be, just because it takes so much space for the backups.

As spydez points out - ZFS does offer a major benefit for machines (laptops) running SSD: "[it evens] out write activity to avoid hot spot failures [in flash]."

http://blogs.sun.com/jonathan/entry/not_a_flash_in_the

You also have to think about the possibilities that would have been possible with ZFS, like hybrid storage pools: efficient use of a Flash + Conventional HD setup

http://blogs.sun.com/studler/entry/zfs_and_the_hybrid_storag...

SSDs already wear level internally; ZFS makes no difference there. L2ARC would have been nice though.

Apple just announced a press event on 9/9/9. Which is 6/6/6 upside down. This is an important number in UNIX filesystem design. I feel something ZFS will happen on that date.

Don't give the analysts any ideas.

Simplest explanation: couldn't be finished in time to a sufficient level of quality, therefore it was deferred for a later release.

Second simplest explanation: licensing issues.

Both of these explanations are very very boring, and I haven't seen anything to suggest that any other plausible explanation is very interesting.

The simplest explanation doesn't hold much water. I never heard of someone losing data on ZFS but there are whole businesses based on the certainty people will lose data on HFS+. It could be because all the people using ZFS today know very well what they are doing and the same can't be said about the average Mac user, but, still, ZFS seems rock solid.

ZFS the file system, probably. Putting a reliable and fast implementation of ZFS in the OS X codebase may be a different story, as is working out a good UI for migrating users. ZFS is pretty complex, I wouldn't shrug off the challenge of implementing it well.

Well, the BSD port of ZFS has the reputation of being less reliable and difficult to administer due to lack of certain features and tool support, so possibly Apple ran into similar problems. The NetApp lawsuit can't help, either.

Really? I hadn't heard that. What are the problems (especially reliability ones) with ZFS outside Solaris? I always assumed it was more or less a straight port. ZFS doing its own RAID implies to me that it sits relatively tightly on top of a simple block device layer.

The NetApp issue, though, does seem plausible to me. At the very least, you'd want to see some level of indemnity from either NetApp or Oracle before shipping this on millions of devices.

For a while my FreeBSD 7 ZFS FS would be empty every time I rebooted -- as if it was reformatted in the boot scripts. That was a pretty big bummer and put me off of using it seriously for a few years. The Solaris ZFS bug was enough FUD for me to wrap it up entirely.

I haven't heard anything specific, so certainly take it with a grain of salt. I was considering using ZFS on my home backup server a while back but didn't want to deal with OpenSolaris, so I did some Googling to see if the BSD port was worth it. This was two or three years ago so I don't remember the specifics, but I decided it wasn't worth it and people griping about disappearing data, disappearing zpools, and stripes that acted like black holes.

Apple tends to stay away from half baked ideas. It probably just wasn't ready and the microscopic marketshare of OSX Server would have made a major shift in priorities hard to justify. They never planned to bring ZFS to OSX client in 10.6 so this doesn't really effect many people. I would expect an updated SL compatible ZFS read/write beta on ADC at some point before 10.7

Reply on news.ycombinator.com