Apple can do better than ZFS | Hacker News Reader

Apple can do better than ZFS

devwhy.blogspot.com

55 pointstragiclos27 comments

Having used ZFS on my servers for a few years, it's a bit disappointing that it's not available on my Mac OS X desktop.

Not necessarily because it's ZFS, which is no longer state of the art. But because I've come to consider strong cryptographic checksums, free snapshots and trivial administration of storage as a bare minimum of what's required in a filesystem.

While ZFS may not be completely suitable for a consumer desktop, it is available now. Plus with the hierarchical storage features, imagine how wonderful it would be to have your multi-TB disc accelerated by a few GB of flash on the motherboard.

It's taken ZFS a long time to get where it is in terms of features and stability. It may be over five years before Apple can produce something tried and tested enough to be their new default filesystem.

I really wish someone would come out with a BSD/MIT licensed filesystem, so anyone could use it. It wouldn't have to be the one with the most features, mainly simple and good at storing typical users' files. Currently, it seems FAT32 is the only thing out there that is able to be implemented without having to worry too much about patents, licenses etc (with some exceptions). The only issue is, FAT32 sucks! I have a Canon Rebel camera that can record videos, but they can only be up to 4GB because of crappy FAT32. It doesn't matter if I have a 32GB flash/CF card in there, they can still only be 4GB.

There are some great BSD/MIT licensed filesystems. The BSD's UFS is as good if not better than NTFS/EXT3/HFS+. It isn't cross platform due to a combination of being complicated, each systems being slightly different and lack of market share.

You are high as a kite. UFS is more comparable to ext2, except with more data loss and not even a pretense of a stable disk format even within the same platform!

that's a textbook example of one of the things pg says not to do here: "When disagreeing, please reply to the argument instead of calling names. E.g. 'That is an idiotic thing to say; 1 + 1 is 2, not 3' can be shortened to '1 + 1 is 2, not 3.'"

http://ycombinator.com/newsguidelines.html

you often have knowledgeable things to say, but that's not enough around here.

You very much underestimate NTFS if you think UFS is comparable. Regardless of what you might think about MS, NTFS is a great filesystem (and Windows doesn't even use many of its features, such as what HFS calls forks, or using OLE to make the .xxx style file extensions obsolete). Reparse points make solving certain classes of problems very easy too.

See the Microsoft vs TomTom lawsuit: http://en.wikipedia.org/wiki/Microsoft_v._TomTom

"It turned out his research team had about the same number of people working on their FS as Apple had working on HFS, HFS+, UFS, NFS, WebDAV, FAT, and NTFS combined. I think people don't appreciate how productive Apple is on a per-engineer basis."

The thing is that's not quite right. The comparison there is between the crew maintaining file system support versus the crew implementing a new fs from scratch. The Author pays lip service in saying it normally takes more than one OS release to implement a FS, but still underestimates the effort it takes to make a completely new production ready FS.

Yes ZFS is not the be all end all. BtrFS, distributed FS's like Hammer and Venti, as well as the occasional complaint about ZFS are exhibits A-? for that, but that doesn't mean Apple can create an FS of the same sophistication in the 18? months that they release in.

That Apple has not yet done so is unexplained.

My explanation is that filesystem performance on desktop and laptop systems is not that big a deal; while on larger servers like what ZFS is targeted for, filesystem performance in all facets is very important.

I'm running ext4 on my laptop. It is significantly faster than ext3. Cold Boot time is awesome. (Sorry HN - I should really test/validate/quantify this, If I get some spare time - I'll try it out and post it here)

Don't forget stuff like time machine is tied into some of Apple's FS hooks.

And of course the fact that the high end (apples market) is undergoing a pretty major shift in hard drive technology at the moment (SSD's).

hfs+ bears the marks of some decisions that made sense in the eighties, when macos was barely capable of running a single program at a time, but are limiting today. pretty much everything requires locking the volume catalog, which makes disk access almost single-threaded. add to this the fact that apple is not afraid to throw out old stuff and bring in new -- they chucked the entire operating system nine years ago -- and the fact that they are still using hfs+ seems very strange indeed.

the only thing i can figure is that they were grooming zfs as a successor, but it was taking longer than expected. now i guess they have to go back to the drawing board and think of something else.

"they chucked the entire operating system nine years ago"

It's not like they built another OS - NeXT had one ready to go - even ported to x86. IIRC, all previous attempts to write a new OS to succeed MacOS classic failed miserably.

What advantage NeXT brought is that it's a very modular OS. Unix programs, traditionally, don't look the OS from too close and that makes swapping file-systems easy. Or, at the very least, possible.

Oh but it is - Macs are often used for, say, video editing. SGI realized the importance of filesystem performance in this application space and developed XFS (which Apple could adopt easily) and GRIO.

It would be intensely difficult technically for Apple to port XFS (perhaps not as bad as the rewrite to make it actually use Linux's VFS instead of implementing its own). It's also GPL — never going to happen for that reason alone.

As a longtime XFS user, I can tell you that it has pathological problems when used anywhere near a user's desktop: unlinking is abysmally slow (and even worse under rm -R). It also has a habit of delaying flush for way too long, both using up piles of kernel memory and making itself extremely vulnerable to power loss.

Ah yes, I'd forgotten it was GPL. Back to the 'ol drawing board...

I've an Octane at home, which is my main machine except in the summer when it makes my flat too warm :-)

"It's also GPL — never going to happen for that reason alone"

If SGI has the copyright, they can still relicense it for use under OSX. I don't think Rackable would block this idea.

SGI has the full copyright to the version they initially released, but modern XFS has spent 8 years getting integrated into Linux by many independent contributors.

Oh please, apple is the worst file-system designer on the planet. HFS, HFS+ and what not. Wastes tonnes of space (space allocation+journalling) and i'd rather use FAT, atleast it is compatible with other operating systems.

Apple should stick to fancy buttons and animation thanks.

They are hiring a FS guru so if you know any good ones try to get them to apply and make the next one better.

Hindsight is 20:20. HFS was release in 1985!

"filesystem performance"

Although performance is a nice feature, the most important things in ZFS are the zero-cost snapshots (very nice for backups, versioning etc) and redundancy and data corruption prevention. I love each and every file on my drive and I want their contents correct.

There is HAMMER filesystem by DragonFlyBSD team http://www.dragonflybsd.org/hammer/

just take it and improve it :)

ZFS is several years old now. I'd think someone could implement a filesystem that borrows many of ZFS's best concepts and adds significant improvement. I haven't kept a very close eye on btrfs but I hope it meets this since it seems poised to take over as the new default filesystem in the next few years.

I just don't get it why age matters.

If something is stable and good enough, then why should it be reinvented, instead of improved?

in the case of file systems, 'improving' around the original design decisions is often impossible. take hfs+ for example: almost all the filesystem metadata is stored in a single b-tree called the catalog. doing almost anything with an hfs+ volume requires exclusive access to the catalog, which locks out all other callers for the duration of the file operation. the solution is to spread the metadata out to other parts of the disk, but that can't be done without breaking hfs+ compatibility.

Oracle owns BTRFS, Sun owns ZFS, Oracle owns Sun.

There's only one way this can play out.

That theme/website looks really good. Stunning.

I'm kind of off-topic, sorry about that guys.

Reply on news.ycombinator.com