How ZFS continues to be better than Btrfs.
rudd-o.com
rudd-o.com
The article is also full of irrelevant details and downright faulty statements.
ZFS is more stable. Yeah, if it wasn't that'd be really bad. Keep in mind that when ZFS was new it wasn't particularly stable either, despite the arguments put forward - a file system must mature with time.
ZFS isn't for the typical enthusiast/home-user. It can't be more apparent than the fact that ZFS dosesn't have a fsck-tool, why doesn't ZFS need a fsck-tool? Because every time you'd need it you just restore your backups from tape instead ( Should ZFS have a fsck tool? http://www.osnews.com/story/22423/Should_ZFS_Have_a_fsck_Too... )
You can yank the power cord of your machine -- you will never lose a single byte of anything committed to the disk. ZFS is always consistent on disk, and never trusts any faulty data (that might have become damaged because of hardware issues).
Nope. Never mind that ZFS assumes that harddrives works to spec, which consumer drives do not (keep in mind that ZFS is designed to be able to run on consumer drives). Meaning that you can not yank the power cord to your machine (if you care about your data).
ZFS truly is awesome, it makes all other easily available linux filesystems look really, really, really poor. Which truly sucks, if you use linux (honestly, I haven't checked the linux alternatives for ZFS but I have a really hard time believing that anyone should trust such an untested solution, just use a stable linux filesystem and wait, patiently, for btrfs).
So, along comes btrfs that mimics the feature-set of ZFS but brings along many advantages (much thanks to it being developed much later). And btrfs has a different target group. Btrfs is much better adapted for home use (it even has a fsck-tool!).
But, btrfs isn't stable (and it shouldn't be trusted in the near future)... And I just can't fathom how you can write such an comparison between btrfs and ZFS today, it just doesn't make sense.
When btrfs is stable and mature things should get interesting.
Oops, actually back in 1995 when XFS was made available, it hadn't any fsck because it supposedly didn't need it. However, they finally made "xfs_repair" available one year later, which while not being called "fsck" proper, still is the same thing.
BTW one thing that xfs_repair did for me was getting back most of the data from an xlv array with a failed drive. Isn't it awesome?
It actually is a decent read: Should ZFS have a fsck tool? http://www.osnews.com/story/22423/Should_ZFS_Have_a_fsck_Too...
* ZFS doesn't include fsck because it doesn't make sense when your FS is guaranteed to be consistent. fsck is for when writes go bad because the window is there for that to happen.
Your claim is like saying a an electric car doesn't make sense because how is the consumer would be confused that it doesn't have a gas tank.
You don't backup ZFS filesystems to tape. I don't know of anyone doing that. It's a weird statement. If you want worst-case-scenario backups, you replicate.
Of course with auto-snapshots setup, and the basic inability of ZFS to become corrupted (IME, never had an issue with v28), you wouldn't be more or less likely to do that as a consumer than you would be to buy an external HDD for your Mac and use TimeMachine.
The only reason it isn't for the typical user is that the typical user doesn't run Solaris or FreeBSD.
* "Never mind that ZFS assumes that hard drives works to spec"
I don't know where you got that. It's actually just the opposite. It checksums everything and includes several facilities to catch early corruption and address it before you ever lose data.
On top of that most ZFS arrays I've worked with are standard 1TB to 2TB SATA systems. Nothing particularly special about the drives. Early on that was one of the big selling points of ZFS. That it was built for commodity hardware.
I personally have pulled the power plug on 24 to 48 disk arrays, literally hundreds of times while under heavy load and have never once lost data.
Not that I haven't run into other issues a few years ago, but data loss due to sudden power loss wasn't ever one of them.
It's almost comically easy to lose data (again, IME) on an ext3/4 system under the same scenarios. That's comparing zero ZFS losses to maybe a dozen of ext losses over the past 3 years.
In the lab, yes. Oracle killed my plans of adopting it.
I don't know where you got that. It's actually just the opposite. It checksums everything and includes several facilities to catch early corruption and address it before you ever lose data.
Yes, that is the idea. But it is based, as the article mentions, in ZFS uses atomic writes and barriers. So, how can you guarantee that? ZFS does it by writing data to the drive and making sure that the data actually has been written, how can ZFS be sure of that? It asks the drive. The problem is that consumer drives often lie and report that data has been written when it is currently only residing in the cache. And that's a great recipe for data corruption.
ZFS checksums are beyond awesome, but they don't combat this.
Jeff Bonwick explains it better than I could:
As you note, some disks flat-out lie: you issue the synchronize-cache command, they say "got it, boss", yet the data is still not on stable storage. Why do they do this? Because "it performs better". Well, duh -- you can make stuff really fast if it doesn't have to be correct.
Before I explain how ZFS can fix this, I need to get something off my chest: people who knowingly make such disks should be in federal prison. It is fraud to win benchmarks this way.
http://mail.opensolaris.org/pipermail/zfs-discuss/2008-Octob...
I personally have pulled the power plug on 24 to 48 disk arrays, literally hundreds of times while under heavy load and have never once lost data.
I hope you don't use consumer drives or really do your research.
The same problem has bitten people running ZFS in a virtual machine, such as vmware ESX, that had similar problems where they couldn't trust the drive to behave as it was told, and corruption followed. Which kind of sucks when you don't have any means of repairing the damage.
Also, the replies I've gotten seems to imply that I have something against ZFS. I love ZFS but I see more potential in btrfs - but btrfs is many years away from even being worth considering an opponent to ZFS so that is kind of moot. All I want is a decent copy-on-write linux file system with cheap snapshots (and preferably good encryption support).
The ZFS stand on this is that nothing can go wrong. If something does go wrong, which by the way is impossible, you better have your backups ready because you are on your own.
If you are runninga a system at a scale where ZFS makes sense and not making backups of critical data, your operations process is fundamentally broken anyway. ZFS doesn't change the need for backups one bit (and it brings to the table some novel ways of doing offsite snapshots and whatnot, to boot)
Now ZFS always keeps around the latest 3 transaction groups, regardless if any data on any one of those transaction groups has been freed/updated already.
So if the last transaction group gets corrupted during an abrupt power down, it can always go back to the latest consistent one.
Keep in mind that, with one exception, ZFS is pretty much rock-solid from day one.
Also, you should really read: http://www.c0t0d0s0.org/archives/6071- No,-ZFS-really-doesnt-need-a-fsck.html
Citation needed.
And that article doesn't really address any of the problems that the osnews article brings up, it just tries to sidestep the issue by more or less saying that the same fsck tool that ext uses won't work on ZFS. Great.
More importantly, what's the problem with scrubbing instead of fsck?
Followed by that was people having issues with data corruption... ZFS is great, but it does break from time to time (whether it is better than other file systems is subjective when you consider the options of restoring it). Low failure rate is, for many, a weak comfort when data is irrecoverable.
More importantly, what's the problem with scrubbing instead of fsck?
Scrubbing repairs data, not broken file systems. Scrubbing is of course great but it isn't the answer for everything.
Ah, it seems our definitions vary.
ZFS has had problems, but all production filesystems have. From a strict standpoint, what the original author wrote about bugs is wrong, but from a practical standpoint I don't think it matters. ZFS has been as solid as any other production filesystem from the beginning -- which is to say that any production filesystem might one day rear up and bite you. It's just the nature of our profession.
That's just my opinion though. :)
Scrubbing repairs data, not broken file systems.
That depends what's broken. ZFS has ditto blocks (multiple copies of the same block) going up the tree; roughly speaking, the higher up you are, the more there are. The tree and metadata has more protection than the data itself.
If, say, all the ditto blocks and all the mirrors are corrupt, reconstructing that is going to be hard. I think a case needs to be made that an fsck tool would do a better job repairing than scrubbing, which is non-obvious to me.
Maybe you should provide some citations yourself first before asking citations from others.
"Keep in mind that when ZFS was new it wasn't particularly stable either"
"ZFS isn't for the typical enthusiast/home-user"
"Nope. Never mind that ZFS assumes that harddrives works to spec, which consumer drives do not"
etc.
When we used it in production about 5 years ago it was not so rock-solid. We had to bring in Sun-Support and went through some nasty downtimes for lengthy resilvering.
Just like SystemTap mimics the feature set of DTrace and brings along many advantages due to its later development: http://dtrace.org/blogs/ahl/2007/08/02/dtrace-knockoffs/
However, I can only run btrfs on Ubuntu natively. Therefore, to me, btrfs is better than ZFS.
I have a NAS that uses mdadm now (it did save me when one disk went completely bust), but RAIDZ sounds even better.
ZFS is great, I'm sure. But btrfs is great too and rapidly improving. I don't think the Linux world has much to worry about.
Right, it's a big conspiracy.
Obviously this also insulated Solaris from any of the drivers in the Linux tree, so 4.5 years later it still doesn't run on anything but custom designed hardware and a handful of vanilla x86 server configurations.
The CDDL was a great harm to the broader community. Like I said I don't see that there's much argument to be made there.
(edit rather than response to avoid prolonging a discussion: obviously Bryan was there and I wasn't, but it should be pointed out that not everyone who was there agrees with that take, nor are our points exclusive. The wikipedia page on the CDDL has a reference ([6]) which jives with my understanding at the time: http://en.wikipedia.org/wiki/Common_Development_and_Distribu... )
So yes, we rejected GPLv2 -- but it was not because we were afraid of becoming an organ donor to Linux, but rather because it would have overly restricted the freedoms of our community. In this regard, we were forward looking: it is now broadly accepted that the GPLv2 is an anti-collaborative license[1], explaining its acute decline for new work.
[1] http://dtrace.org/blogs/bmc/2012/08/01/post-revolutionary-op...
Broadly accepted where? You certainly draw grand conclusions from head-bobbing at a talk you made.
It's the licence used for I dare say the largest collaborative open source project in the world, Linux. GPL is the most widely used open source licence, used in tons of collaborative projects, from the top of my head: gcc, git, mercurial, qemu, ffmpeg, x264, blender, gimp, inkscape, mplayer, emacs, etc are all examples of other collaboratively developed GPL licenced projects.
And how exactly would it be 'anti-collaborative'? If anything it's a great licence for 'collaborative development' as all participants are legally bound by the licence to release their changes in source form when they distribute.
The rest - agreed.
I see quite the contrary.
Any license that allows you to release your derived work as a proprietary and closed-source decreases the amount of collaboration because now nobody else can collaborate on your proprietary fork. One of the reasons for, say, IBM not to give part of AIX to FreeBSD, is that they fear, justifiably, HP may take their collaboration and incorporate it into HP-UX, giving it an advantage over IBM's proprietary product. If IBM incorporates some übercool part of AIX into Linux, HP cannot use that to benefit HP-UX. Fear of becoming an organ donor is lessened because the receiver can't run away with your liver.
In the same way as GPL did, no doubt. FreeBSD is chock full of CDDL tech, but can't use GPL. Illumos can use *BSD code too, but not GPL.
This is back to the BSD vs GPL debate. That's just, like, your opinion, man.
Sun drops some really nice tech out in the open, which is the only reason Illumos can exist (and is now firing on all cylinders). FreeBSD nabs many of the good bits to good effect, as does OSX. And all some people do is complain. Bizarre.
Obviously it was better for Sun to release OpenSolaris under the CDDL than not. But it was a very poor choice, and again in my opinion a great harm to the community.
And frankly I don't see the Linux people complaining about anything. The linked article was written by a ZFS proponent...
I think the distinction is academic. In practice FreeBSD can use CDDL code, but not GPL.
And frankly I don't see the Linux people complaining about anything.
Every time some piece of Sun tech comes up on a nerd site or aggregator, a licensing flamewar erupts. No matter how cool or useful that piece of tech is, some twit beats the CDDL vs GPL horse once again. Typically this argument then dominates the discussion. Sometimes it's the only comment. Every time.
It's the ultimate manifestation self-entitled bike shedding, because the doers have more interesting things to do. What I find repulsive is that the doers have made cool blue Ferraris for free, and we have giftzwergs complaining that they aren't cool red Ferraris, which is demotivating for the people creating all this. How would you feel if you created something neat, gave it away for free, and were thanked with a wall of people complaining about your choice of OSS license?
I apologize if I come off unfriendly. It's just that I'm interested in the actual tech, and this noise is really growing old.
I'm sorry you're interested in "actual tech". But in the real world stuff like the legal ability to use software sometimes gets in the way of our geeky aspirations.
But it doesn't help when "great harm" hyperbole gets thrown around. The world has benefited from the gift, unless anyone is daft enough to argue that Illumos' existence, FreeBSD's incorporation of ZFS and DTrace, and OSX's incorporation of DTrace, are a bad thing. Hell, DTrace is even being used for PS3 game production.
Linux can do the same as easily, they "just" have to relicense the kernel in BSD.
Please stop trolling.
There is simply no equivalence with the GPL vs. CDDL. Neither license permits redistribution at all when combined with the other. The only way to do this would be to, as you say, "relicense" the kernel by getting every copyright holder (there are tens of thousands by now) to redistribute their code under the CDDL.
I used to work on this stuff. I used ThinkPads and Dell laptops bought from the store, random white boxes I had lying around, and a particular Dell workstation. Custom designed hardware? Oh, and btw, my ThinkPad had a wireless card that was supported by Solaris. Never gave it much thought, except when I switched jobs and installed Linux, which at the time didn't have the particular driver...
It does not require custom hardware, it runs on any server.
Oddly enough, I have a laptop that runs OpenSolaris better than it does Ubuntu. 12.04 wouldn't even find the ethernet controller...
It's not a conspiracy, it was deliberately released with a license designed to keep it out of the Linux kernel.
It was painless to install and so far has been reliable. I'm not yet trusting it for primary storage though.
Unfortunately it's now controlled by Oracle so you're probably lucky that you can use it without hiring an 'storage consultant' for $100k+ a year.
However, your second statement is blatantly false. Much of ZFS development happens in the open in the illumos and FreeBSD communities. Joyent and Nexenta base their business on ZFS and pay people to work on the open source illumos.
(Disclaimer, I used to work for Nexenta).
Best install I have done for some time.
Sorry, but Linux isn't the end all be all open source operating system, and is the last thing I will put into a production system where I care about my data and or stability.
well dude, here's a fact: http://lwn.net/Articles/506244/ have you even used btrfs lately? because that collides with some of your "facts"
so sure, zfs has advantages over btrfs and is arguably better right now, but the latter is still experimental. the main reason for btrfs is the proper licensing for the Linux kernel, anyway - else we'd all be hacking on zfs since day one and there wouldn't even have been a need for btrfs.
[1] http://www.youtube.com/watch?v=-zRN7XLCRhc#t=46m45s
[2] http://blog.delphix.com/matt/2011/11/01/zfs-10-year-annivers...
[3] http://blog.delphix.com/matt/2012/07/11/performance-of-zfs-d...
To me, ZFS and Dtrace are of no consequence if I can't use them on my own hardware. And I am sure there are many more people like me.
I have tried and given up running Solaris derivatives on bare metal even if for running just command line. The hardware /peripheral support just doesn't seem to be there.
Other setups can work, but you need to be very aware of this: http://illumos.org/hcl/
Much better than the Linux machines that were replaced.
>> there are many more people like me
Recently I was wondering how can there be so many tech startups around just now. Nothing under the sun is new so how can they cut a living?
Ive realised this attitude (which there is absolutely nothing wrong with) is the reason. The established players leave gaps for their own reasons and the little guy comes along and seizes the opportunity.
Skipping over cheaply accessible best in class technology, at a philosophical level at least, seems akin to pointing a 12 gauge at your toes.
tl;dr: absent glaring port issues, ZFS on Linux is or should be at maturity of ZFS itself -- which is to say, very mature.
ZFS might be great if you're running a file server where you can pack many gigabytes of memory into a server and not use it for anything else. But if you are trying to run a cloud or VM server, where you are trying to pack lots of jobs on a single machine, the cost/benefit ratio of a copy-on-write file system (whether it is btrfs or ZFS) may not be worth it.
Sorry, but having your underlying storage be rock solid is absolutely crucial for VM's.
This is the same reason why Linux doesn't have a viable DTrace competitor: the “what's wrong with kdb?” attitude started with Linus and it took way too long for the near-unanimous sysadmin voice to be heard.
Fortunately, there are types of vdevs other than RAIDZ, and using SSDs for the ZIL can improve things as well.
http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuni...
... but some are strange. why should a filesystem implement a cifs export? ok, zfs does it. but why should it?
# zfs set sharesmb=on export/home
This is no longer true. As of Linux 3.6, btrfs has the ability to apply quotas to subvolumes.
I went to a half-day Suse brunch event 2 months ago and got myself a eval copy of SLES v11sp2, just to check out brtfs. I tried to be mindful of the "it's not ZFS" attitude, and gave it a lot of leeway. Without going into the maturity of the code bse (I have no idea how stable it is), I found the capabilities and commands a bit cumbersome, counter-intuitive, and the FS as a whole just plain lacking. The article summed it up much more succinctly than I ever could.
I've been using ZFS on my primary workstation on a daily basis since the early beta days of FreeBSD 8.0 (Jun '09, I believe), and it has been a pure joy to use. ZFS v28 (the last released version from Sun/OpenSolaris, I think) found on FreeBSD 9.x is rock-solid and has reasonable performance (as compared to the speed and memory problems seen in in earlier releases).
I've put the filesystem through its paces, doing things that are rightfully called foolhardy, and I have never lost data. I've painted myself into a corner a few times (I curse the fact that you cannot remove hardware from the pool, as with Linux or AIX LVM), but never have I lost data when redundancy was in use. I have had bad sectors crop up on a single-disk zfs pool (single-copy of data, should have known better), and a "scrub" let me know exactly which files had incorrectable problems. Heck, I routinely flip the switch on my PSU when I'm too lazy to walk around my desk to do a proper shutdown, and it comes right back at next boot, no lengthy fsck, no data loss. I usually scrub monthly to check for errors, and otherwise I don't do much maintenance.
Right now, I have 6 2TB Seagate "green" drives in a raidz2 array (raid5-like with 2 parity disks). One disk has been removed due to errors, so I still have a spare disk to tide me over until I can afford a replacement. It runs very well.
I wish native ZFS crypto was available. Right now, each physical device is fully-encrypted with FreeBSD's "geli" (customer data, tax records, etc.), so ZFS doesn't even really have full domain over the physical devices, which makes it even more impressive that I've never lost data.
I will say that enabling compression and/or dedup will cause performance issues. I've got an 8-core AMD processor (XF-1850 -- hardware AES support, yay!) w/ 16GB of DDR3, and I get interactive desktop lag when heavy I/O is going against compressed and/or de-duped data. I don't know if this is a FreeBSD scheduling issue , or inherent with ZFS (any Solaris users want to comment?). Since it was mostly just to test (not like I need the space savings), I just do without.
In any case, I hope Linux's ZFS native kernel port stabilizes. brtfs just isn't there yet.
Of course no filesystem could have survived that. But I still could really have used a salvager (fsck). I actually recovered a couple of critical files by dd-ing blocks off the disks.
As a member of the illumos community, I long ago turned the question around: given ZFS, DTrace, Zones, and the rest, all available in a free (as in speech) operating system, why would I use a different system that prohibits incorporating all this valuable free software?
It depends on perspective. I think it's ridiculous that someone would even put out a whole distro that's free and useful, and I am really grateful for that.
> why would I use a different system that prohibits incorporating all this valuable free software?
In my case it's simple: I don't have the time or energy to put out a distro that conforms to everything that I want.