Ubuntu 20.04’s zsys adds ZFS snapshots to package management
arstechnica.com
arstechnica.com
Has anyone had any real problems with apt install or apt remove busting their systems? Seems more like a solution in search of a problem.
* For expert users who have hosed a system -- particularly if it happens to be a production host -- there's an immediate sense of relief and ability to rollback safely, confidently and quickly.
* For less-experienced users who run into problems, there is a convenient and straightforward way for their technical support to explain to them how they can restore their system to a previous state.
Since hosts and their installed packages have varying hardware and configuration states and update/maintenance schedules, incompatibilities and breaking changes are likely to occur occasionally and that's where snapshots can help.
I’d think the more useful feature would be taking a snapshot at the beginning of the apt run; and then tossing it away if apt manages to complete successfully. That way, apt returning a nonzero exit status could be caught by a wrapper script, that would then transactionally revert the changes.
I believe NTFS still does this for Windows updates, even if “transactional filesystem” features are no longer “available” (i.e. exposed to userland.)
Of course, unlike NTFS’s filesystem transactions, ZFS snapshots aren’t MVCC — you can’t have multiple parallel ones going on at once that get merged on success. So that apt-wrapper-script reverting, might throw away something else you’ve done to your rootfs (and hopefully only your rootfs; such a technique would effectively require moving all user-mutable data to other mounted volumes.)
> and hopefully only your rootfs; such a technique would effectively require moving all user-mutable data to other mounted volumes
I believe that zfs installation creates a separate dataset for /home, meaning that it won't be rolled back along with everything else. The same should be true for things like /var/lib/, but I haven't done such an install in a while, since the first 20.04 build with zfs enabled.
Incredibly frustrating if you don’t know about it, and incredibly helpful if you do.
Creating ZFS snapshots is almost instantaneous due to the CoW nature of the filesystem. Is it deleting old snapshots during that process? If so, I'd suggest checking to make sure you have the async_destroy feature enabled for your pool.
You have a snapshot to go back to in that case. It saves you exactly from power-loss kind of situation.
Yes, but even so it is far rarer than being frustrated waiting for f'ing grub update.
* https://docs.oracle.com/cd/E86824_01/html/E54764/beadm-1m.ht...
Understandably some people may be more interested in not losing their data than the licensing headaches it may come with, so ZFS it is.
Some, like yours truly, run FreeBSD on their NAS, where there are no license issues either, and are very happy it rebased on OpenZFS recently.
Btrfs has been working fine for years and is used largely at Facebook. RAID 5 and RAID 6 are still unstable (and marked as such) but the rest just works.
I get the feeling that if you are a ZFS user then RAID 5 & 6 not working is just a deal breaker.
Btrfs also has some advantages for personal use when you compare it to ZFS: it supports combining disks of different sizes and is easier to grow.
I think that's wrong. A lot of 4 drive models are in stores end users shop.
Let's not truncate this sentence to avoid altering its meaning. I certainly don't deny four drives NAS exist but I think dual bays are still a lot more common.
Anecdotally I know a lot of people with one drive NAS (some ISP provide them here), a few people with two drives NAS (including me) and no one with four drives (but I don't know anyone heavily into photography or video which seem to be the main usage).
My point was that you can still address a large segment of the home users market without RAID 5 and RAID 6.
The RAID5/6 issues are significant, but don't pretend that Btrfs would be OK if these problems were resolved. There are still many others waiting to be fixed as well.
Btrfs was originally introduced in Linux 2.6.29 in March 2009. It is still flakey in many situations.
ZFS was first release in Solaris 10 6/06 in June 2006, with it being open source almost right away. OpenZFS was forked in August 2010.
Should Btrfs at some point simply be declared a sunk cost and something new be created? Because from the outside it looks like it's hardly gaining any traction because of its reputation for eating data (maybe only with its RAID code, but still).
That's why btrfs RAID5/6 mode is in the sorry state it is. Nobody capable of paying the piper is interested in it. It is a domain of datahoarders, but that's not where the money is. Even Synology has put btrfs on top of mdraid for that.
Outside the parity modes, btrfs is stable and reliable. It is here to stay.
[1] https://www.zdnet.com/article/why-raid-5-stops-working-in-20...
ZFS will never be part of the Linux kernel due to both licensing issues and it including its own crypto, cache and other subsystems.
Personally I have high hopes for bcachefs, but Btrfs is here to stay.
btrfs was removed from RHEL because RedHat themselves don't have the time or energy to support it or support systems using it. That's going to hurt adoption. Ubuntu is working actively towards ZFS as an available/default option, and once it is there's not much point in using btrfs unless you have specific use cases. Everyone else I know either uses ZFS or XFS in production. For my systems, I would definitely not trust btrfs with critical data.
Here's the thing: btrfs is 'fine', but it's not 'good'. It's the only filesystem since 2001 where I've suffered unrecoverable filesystem corruption as a result of a power outage (on a fresh 18.04 installation last November) and had to salvage data and reinstall.
The tooling, in a lot of cases, does the opposite of ZFS and touts it as a feature. For example, snapshots are not recursive and they say that this is great for if you want to snapshot / but not /home; this is true, but it also means that I cannot subdivide datasets.
For example, on ZFS I divide up /var/lib/mysql/ into /var/lib/mysql/tablespace/ and var/lib/mysql/logdb, for example, since both of those have different access patterns and block sizes (16k and 128k respectively). ZFS lets me do this to optimize reads and writes, but I can also atomically snapshot them both. btrfs doesn't let me do an atomic, recursive snapshot at all (and I'm not sure if it can handle different block sizes per subvolume either).
Basically, ZFS is a better, production-ready, enterprise-proven, and surprisingly intuitive filesystem and volume management system, and has been for 14 years. btrfs has weird tooling, incomplete features (which they don't just remove from the roadmap for some reason), data loss issues, and the only large-scale use I've heard of it Facebook, who also work on it (easier to use btrfs when you've hired all the experts).
With Ubuntu shipping ZFS enabled by default, most reasons to use btrfs go away; it's not mainlined in the kernel, but that doesn't matter if your distribution has it enabled and supported by default.
Like Mir, Unity, Upstart and all the other technologies Ubuntu adopted, but removed after a while again? I always have the feeling that the Canonical engineers try to advance the Linux ecosystem by introducing new technologies, but somehow always pick the wrong ones, which end up not being adopted anywhere else. Granted that's different with ZFS, as ZFS already has a large user base, but there is nothing guaranteeing that Ubuntu won't remove ZFS in a few years again.
It had a long run, and it’s still more usable than gnome3 for power users. Unity wasn’t successful but wasn’t deficient.
Upstart wasn’t as good as systemd that cane after it or daemontools that came before, but it WAS an upgrade compared to the knit scripts.
Ubuntu/canonical are not afraid to try, which I think deserves more credit.
They don’t have the clout/pockets to push less popular things like systemd the way RedHat does, until all the problems are sufficiently minor.
Oh, and snap and Mir we’re both attempts at grabbing control and both deserve the scorn.
The other projects were (for the most part) anonical originals" that they tried to create -- primarily on their own -- to replace existing products. ZFS is going on 15 years old, quite mature, highly trusted, and, of course, it already has a huge existing userbase.
It's certainly not an internal Canonical project that requires them to devote huge teams of developers to it. Unlike the others you mentioned, ZFS will continue to exist and grow regardless of Canonical's involvement.
Oof, "Canonical originals", that should have read (not sure what ate the two missing characters, HN or the iPad).
Is still used in some maintained RedHat distros, I assume this burns people like you each time is mentioned.
Probably because
>> I assume this burns people like you each time is mentioned.
is rude and borders on a personal attack.
FWIW, SUSE is the Enterprise Linux vendor that has went "all in" on btrfs.
---
Two semi-related, personal thoughts:
1. btrfs today feels a lot like reiserfs did (going on / nearly) 20 years ago.
2. I expect Oracle to -- at some point -- integrate ZFS into Oracle (Enterprise) Linux. That seems like a great way for them to win over some "converts" from RHEL.
---
> Oracle owns ZFS. Or, rather, it bought Sun, which was the company which originally wrote ZFS. So, it doesn't really need to worry about the licensing issues.
Oracle owning ZFS doesn't magically absolve them of risk however it does reduce the risk. But with ZFS being open source, another contributor might have a claim if Oracle were to breach the CDDL license with code written by said contributor (similar to the SCO vs UNIX lawsuits of yesteryear).
To get around this Oracle would need to contact all 3rd party contributors and get the to agree to a license change. This would, in all practicality, then be reflected in a new release (like the GP suggested).
However it is now even more complicated than that because OpenZFS (which is what Linux runs) has diverged from ZFS (which Oracle maintain). So while Oracle would need to agree to any changes in the licensing of OpenZFS (as there's still Oracle code present in OpenZFS), Oracle are not the ones maintaining it and nor could they ship their own version of ZFS as Linux kernel modules to answer any concerns with either OpenZFS licensing nor compatibility with other Linuxes running OpenZFS.
If Oracle wanted to ship ZFS on Linux, they would presumably port over their own version and never involve the OpenZFS team. However, Oracle created btrfs, and seem content to stick with it.
OpenZFS is a fork of ZFS rather than a reimplementation so I'd be very surprised if there wasn't any Oracle code in it (unless you're making a distinction between Sun and Oracle but legally speaking that would all still be owned by Oracle, hence why I didn't make that same distinction myself).
> If Oracle wanted to ship ZFS on Linux, they would presumably port over their own version and never involve the OpenZFS team.
Porting in this case isn't a simple task as there's a lot of "Solarisims" in ZFS that had to be worked around with the Linux port (ZoL -- ZFS on Linux -- didn't happen over night). Plus, and as I'd said earlier, ZFS and OpenZFS have diverged in terms of supported features so a ZFS volume wouldn't be compatible with an OpenZFS volume. I guess this wouldn't bother Oracle since, like most orgs of that size, they have no qualms with creating vendor lock ins. But it certainly wouldn't do much to sell Oracle Linux to the wider ZoL community.
The reason the distinction is important is that Sun released that code under the CDDL. If Oracle wanted to somehow claim it couldn't be used, it would require a massive lawsuit that would basically be against the concept of free software.
I don't understand why that would be necessary. CDDL (and equivalent) licenses don't transfer ownership of IP (intellectual property) to the public domain, they only outline usage and distribution of the code and compiled artefacts. When Oracle bought Sun, they acquired Sun's IP (amongst other things) -- hence why they could go after Google with regards to their Java IP despite Java being a Sun Microsystems creation. ZFS isn't any different, it might have been distributed under a CDDL license but since Oracle now own the copyright for ZFS code they can legally re-license that code base under another license if they wished. So Sun doesn't factor into the equation any more since what was Sun's is now owned by Oracle.
It's worth noting that you do occasionally see open source projects change their software license years after their initial release. So this isn't an untested theory.
From my understanding of open source projects released under a new license, they often can create a similar fork at the license change time where a version exists under the old license too. The exception is if they use the GPL version that allows updating the license
Nobody is suggesting they retract older version. In fact the exact opposite has been said on multiple occasions: any new license would be signified by a semver (semantic version) bump. So you're arguing against a point that was never made in the first place.
> getting rid of OpenZFS altogether
OpenZFS code would be unaffected because it's forked from a version prior to the semver re-license bump.
> From my understanding of open source projects released under a new license, they often can create a similar fork at the license change time where a version exists under the old license too.
Fork or semver bump. Which is exactly what I was talking about. Also notice that what you described there contradicts what you said would happen ("getting rid of OpenZFS") in literally the same comment.
---
If Oracle own the copyright then Oracle can re-license. They'd have to bump the version number and the previous versions with the older license would forever remain CDDL. But legally they can do it and it wouldn't affect OpenZFS licensing (aside that OpenZFS might then no longer be able to incorporate ZFS code -- assuming ZFS's new license is incompatible with CDDL. But things start to get very murky down that road.)
My argument has been that Oracle can't do this the entire time, but that's why its unfair to say there is Oracle code in OpenZFS. It makes it sound like Oracle has some power over OpenZFS, which is just not the case.
No it wasn't. You kept bringing Sun Microsystems into the discussion and they're not relevant regardless of whether we're discussing ZFS or OpenZFS because Oracle own Suns IP. Moreover, and I can't stress this enough, nobody ever suggested OpenZFS would be "got rid of" except for yourself! You created that straw man argument.
> that's why its unfair to say there is Oracle code in OpenZFS. It makes it sound like Oracle has some power over OpenZFS, which is just not the case.
It's not unfair to say that because that is the literal truth. And I was very clear about not only the distinction between ZFS and OpenZFS but also the limited scope in which Oracle could control OpenZFS (if the OpenZFS guys want to re-license they'd still need Oracle's approval for any Sun IP (which Oracle now own) that remains within OpenZFS code).
Claiming that Oracle code doesn't exist within OpenZFS or that Oracle doesn't still have a say with regards to OpenZFS's licence would be as ignorant as saying Oracle could "get rid of OpenZFS".
I suggest re-reading my initial comment that sparked this discussion. I thought it was pretty clear about the distinction between ZFS and OpenZFS and about how much/little control Oracle have over the licenses of each.
All that aside, one comment you made elsewhere did interest me: whether Oracle could sue Canonical due to copyright grounds with Oracle code in Linux. I don't know GPL well enough to know if this is possible but it's a truly terrifying thought given the amount of contributors to Linux and the prevalence of Linux in propitiatory systems like phones, routers and other places that might contain closed SoC modules.
There is no "just" in "just release it under [another license]". I'd written a detailed rebuttal of why it's not that simple in the GP post to yours: https://news.ycombinator.com/item?id=24331156
Yes when it became apparent that BTRFS was not ready RH did that. It was a slight surprise when it happened if I remember correctly.
I said RHEL v9 OR 10. I guess 9 is unlikely but 10 is a number of years off. If BTFS is a sucess on Fedora and stable then they will surely consider it. That is the point of Fedora...
I would agree with you. Fedora adopting stuff does not mean RHEL will include it, but if it's successful/popular on Fedora then Red Hat will surely be paying attention. RHEL tries to follow Fedora as closely as possible, I would imagine this is a shot for btrfs, but certainly no guarantee.
Btrfs goes though a rapid development; so Redhat has to backport everything to their ancient 3.10 (RHEL7) kernel all the time. That's a lot of work.
Fedora, on the other hand, uses current kernels. They do not have to do this work.
Redhat also cannot choose ZFS while in the license limbo; that's why they push for Stratis.
It is stable, certainly stable enough for enterprise production use according to SUSE.
Also, do these Debian cautionary notes still apply to new code:
> There is currently (2019-07-07, linux ≤ 5.1.16) a bug that causes a two-disk raid1 profile to forever become read-only the second time it is mounted in a degraded state
As for the write-only bug, 1) that only affects 2-disk RAID1 setups where people insist on cowboying around with their degraded arrays, and 2) doesn't result in data loss, merely an additional copy step to restore the array. Don't run your arrays in a degraded state, please.
As I wrote, SUSE considers Btrfs stable for production use, and they're the premier enterprise distro in Europe.
That's easy to say, but if you get a bad disk in a mirror on your boot drives, and for some reason your system restarts during this time period, you boot up into read-only mode--which kind of keeps your server down and kills service availability.
I don't run Btrfs, and so heard about this scenario from Jim Salter (the author of the linked Ars article) in his podcast:
RAID1 is not a backup. Apparently under BTRFS, using RAID1 for HA isn’t acceptable either.
What’s the point of a RAID1 array if you can’t use it degraded? Might as well just go RAID0 too then (you have backups anyway, right?)
Still, I’d love to see more people adopt btrfs because I think diversity in filesystems is a good thing.
I like btrfs because it is just a `cinst winbtrfs` away on win32
or perhaps my impression after this recommendation is wrong and this actually isn't working well?
much confuse
I don't think there is any case where this WinBTRFS would work better than the Linux implementation, but the Linux implementation is solid enough - if BTRFS is good enough for Synology NASes (https://www.synology.com/en-global/dsm/Btrfs), it's good enough for me.
I wonder what led you to this line of thinking though. It reminds me of people being amazed at $NEWER_TECH just because it had less time to accumulate bug reports.
OpenZFS is still open source and available on FreeBSD (amongst others). The licence might not be GPL compatible but the way ZFS is shipped on Linux there isn't any GPL/CDDL mixed code bases (and it's not like there isn't already a long history of GPL-incompatible software licences with Linux kernel modules anyway).
Hence, Canonical (or anyone else trying to re-release under the CDDL) cannot distribute binaries of the kernel modules.
However, you are fully allowed to compile and link the two on your own. This is why the alternative to binary distribution is compiling yourself with the help of DKMS.
As a result, it would be extraordinarily difficult for Oracle to sue since it would not be the CDDL licence terms which were infringed.
That said, boomboomsubban (https://news.ycombinator.com/item?id=24331590) suggested Oracle could sue on the GPL side of things since they have committed code to Linux. It would be interesting if such a position is possible and I suspect that could open the door to all sorts of frivolousness cases (eg closed binaries shipped with Android).
I believe the bug which caused this is now fixed, but who really knows what other nasties linger on waiting to be triggered.
In comparison, ZFS has been a dream. I even moved terabytes of data between a Linux system and a FreeBSD server simply by swapping the discs over and running "zpool import". Superb data portability.