Hacker News Reader

Comment by chronogram | Hacker News Reader

I think it's neat that having the possibility of a ZFS root on Ubuntu exists, but I just don't see myself using it. Will anyone, and if so for what purpose?

I'm not in the home NAS building sphere, but wouldn't you rather have your root be a SquashFS image on an SD card or similar, with the storage section being ZFS?

As for my laptop, I guess the snapshots are nice. I've never had a situation with my laptop where I wished I had snapshots, and even if something did break I only care about the files I have, which I back up. Then the snapshots are an extra safety net I suppose.

View on HN

I use it for two reasons.

The first is generic: I work only on filesystems with data checksumming; the current ones are BTRFS and ZFS, and I have a low opinion of the developers/development of the former.

The second is specific: I keep my disks encrypted; since v0.8, ZFS has encryption integrated, which allows me to have disks encrypted and mirrored without additional layers (BTRFS requires LVM for this setup).

In the past I've also used snapshots, by the way, which are really functional.

ZFS on root enables you to have boot environments, which is a functionality originally found on Solaris. Not that they're in scope for this initial implementation in Ubuntu, but they could be built on top. Furthermore, ZFS protects you from bitrot via multiple copies or RAIDZ or at least lets you know bitrot occurred (scrub). It lets you resize the dataset (I tend to undersize partitions such as /boot). I could probably think of other reasons as well, but this would be enough for me.

> Furthermore, ZFS protects you from bitrot via multiple copies or RAIDZ or at least lets you know bitrot occurred (scrub).

Scrub is not even required for that: all data is checked when read, it's only for checking unused data.

> It lets you resize the dataset

Considering that you are talking about quota on directories, I'm not sure it is really advantage of ZFS. It's basic functionality.

Me and few of my friends have been using ZFS on laptops for few years now (since 2013 on a laptop for me, since 2012 on a workstation).

There's no real increase in power usage except for one thing, which is configurable - the transaction group commit frequency (I leave it default, a friend of mine sets it to very powersaving level coupled with running "nosync").

A quick description is that TXG write period says how often ZFS writes data that was written without syncing to disk, and commits data that was written synchronously. Every "transaction group" is essentially a snapshot, and by default it's done every 5 seconds, or every ~10? MBs written, whichever is first. Data that is written synchronously between that is written into "ZFS Intent Log", and all data written is kept in memory without sending to disk (except for sync writes).

If you disable sync writes (datasets mounted sync=none), ZFS will only wake up drives for writing with the interval set in this parameter, or when you start writing significant amounts of data (where "significant" is also configurable).

If you're willing to lose X seconds of writes, you can configure sync=none and transaction writes to happen every X seconds. My friend does that, myself I prefer my data safer :)

That's the story on power usage with ZFS. I'd recommend maxing out memory where possible, but I recommend that even without ZFS.

Then there are benefits, like recovery from snapshots (more than once I recovered a file I accidentally overwrote because I missed a '>' in '>>'), easy backup/migration with zfs send/recv. I also found it pretty easy to install multiple linux distributions using the same root pool, by giving them different root datasets and putting home directory and similar on separate dataset (on the same pool).

This allowed pretty nice migration between distributions, so long as I kept UID/GID of my user the same between them.

> but wouldn't you rather have your root be a SquashFS image on an SD card or similar, with the storage section being ZFS?

Read-only root filesystems are one of those things that everybody ought to do, yet are usually impractical. Unfortunately, most software doesn't really support the Linux Filesystem Hierarchy Standard, so you have software that modifies itself under /usr (supposed to be read-only), ever-growing logs and caches not stored under /var/log and /var/cache so they can't be managed automatically, etc.

You generally find squashfs roots where the distro is specialized and not general-purpose - for a recent example, see Talos.

Have you checked out NixOS? Everything is read-only and derived from static configuration.

I run NixOS as a daily driver, and while I'm aware that the install media is a squashfs mount and that it's feasible to create server images where /nix/store is a squashfs mount as well. But in day-to-day use on a development machine, running nixos-rebuild switch makes a read-only root impractical, so far as I know.

The OSs that insisted upon scattering application files all over the hierarchy only have themselves to blame for this situation. At some point in the past 30 years thought could have been given to the idea that applications are separate from the platform they run on, but UNIX just doesn't think that way. Everything must be part of the same gigantic Goldberg-esq state so everything can catch fire and burn down at once as soon as there's a conflict of any kind.

Anyway, the fix is pretty obvious: put each application in its own namespace, thus its view of "/usr" isn't the same as the immutable OS "/usr" and it is free to write whatever it wants without causing any trouble.

> At some point in the past 30 years thought could have been given to the idea that applications are separate from the platform they run on, but UNIX just doesn't think that way. Everything must be part of the same gigantic Goldberg-esq state so everything can catch fire and burn down at once as soon as there's a conflict of any kind.

this only happens if one isnt aren't managing applications coherently (e.g binary only packages managed haphazardly, like most linux binary packagers)

on systems where there is a clear base+add-ons distinction (e.g. source based ports), these things happen much less often - all packages are built from a coherent source tree, and read only root is often feasable because packages only install reference files into their respective '/usr/share' area and don't muck with the 'real system' in /etc - the latter being an administrative action (e.g. package installation is a separate action from service configuration / execution)

So basically, all software has to be built from source by maintainers who are careful to make sure it doesn't conflict with other software. And people wonder why applications aren't ported to these OSs.

The OS should be a platform. I should be able to build an application and distribute it to users and have it just run on that platform without involving third party middlemen and regardless of other software they have. You know what OSs accomplished this? The original Macintosh, RiscOS, DOS, and (mostly) Windows 3.1. Why is it that we can't accomplish today something we did 30 years ago?

a platform for what?

a platform for 'building your system', or a platform for 'distributing and executing binaries'?

the binary only distribution mentality comes from microcomputer culture, which postdates unix/mini culture and is rooted in non-technical users installing purchased 'application' programs from vendors rather than computer users (read: programmers) building their systems to suit their own personal data processing tasks. you highlight this in your choice of platforms 'getting this right', but really, these are two different mentalities, cultures, and modes of operation.

not saying one is superior to the other, but square peg, round hole.

and besides:

many proprietary programs for unix just ship with static executables. problem solved.

> a platform for what? > a platform for 'building your system', or a platform for 'distributing and executing binaries'?

These are not mutually exclusive goals.

> the binary only distribution mentality comes from microcomputer culture, which postdates unix/mini culture and is rooted in non-technical users installing purchased 'application' programs from vendors rather than computer users (read: programmers) building their systems to suit their own personal data processing tasks.

"Non-technical". Elitist bullshit. Woz was non-technical? He was clearly a micro guy. People got work done with micros. They made things with micros. Like your glorious programmer master-race, they configured their environment into something that enhanced their lives. They bought and used software the way you import 10000 dependencies. Excel is still by far the most used programming environment in the world, with entire businesses run on tooling never touched by programmer-master-race hands.

Besides, programmers should want it this way too. Which makes more sense to you? A constantly mutating state where changes in seemingly unrelated portions can wreck havok on the whole, or a collection of immutable components which can be swapped in and out at will? A system so complicated it needs sophisticated management software and a database of conflicts and dependencies to keep it all running, or one so simple that 'copy' and 'install', and 'delete' and 'uninstall' are synonymous?

If the UNIX model is so damn great, why the fuck does Docker exist!? If development is so damn great on UNIX, why are people building and distributing dev environments in containers!?

> you highlight this in your choice of platforms 'getting this right', but really, these are two different mentalities, cultures, and modes of operation.

> not saying one is superior to the other, but square peg, round hole.

I disagree. I think one is clearly objectively superior to the other.

> many proprietary programs for unix just ship with static executables. problem solved.

Ha! Yeah. Problem solved. Even for proprietary software with a native Linux version I've almost always had better luck running the Windows version under WINE.

> Everything must be part of the same gigantic Goldberg-esque state so that everything can catch fire and burn down at once as soon as there's a conflict of any kind.

Well, that's exactly the job of competent package managers, to make sure that packages fully stated their dependencies (so that the package manager would forbid you from getting into an illegal state wherein a required dependency was not installed or the wrong version of the dependency was installed), and installed correctly according to the LFHS so that there would be no conflicts.

/opt isn't new. Plenty of people decided that they could throw everything into a folder under /opt, including copies of all dependencies, with no references outside of that folder, and that would be enough. And it might have been enough if it wasn't for sysadmins in charge of uptime tearing their beards out over /opt constantly filling up and causing multi-product outages when it did.

The whole point of the LFHS is that it represented a contract along the boundary between developers (who needed to bundle libraries, save state, have access to a cache, etc.) and operators (who worried about disks and associated costs - why would you ever spend a dime backing up /var/cache? Why would you ever use a fast disk for /etc?). When this contract was followed, systems worked splendidly. When the contract was broken, the people who broke the contract shot themselves in the foot just about the same as anybody writing C code knows how many footguns are in the C language.

Of course, today, with the price of resources having crashed through the floor, and containerization having more or less reached maturity, this is all more or less an academic, historical, moot point. But dynamically linked libraries weren't stupid, they were an artifact of an age where statically linking libraries was horrendously uneconomical and the idea of namespacing anything was enough to get laughed out of the room by people wondering how you would expect to pay for any of it.

> Well, that's exactly the job of competent package managers, to make sure that packages fully stated their dependencies (so that the package manager would forbid you from getting into an illegal state wherein a required dependency was not installed or the wrong version of the dependency was installed), and installed correctly according to the LFHS so that there would be no conflicts.

Package managers are an over-engineered solution to a problem that only really exists if you rely on package managers. Ever had your apt database break on you? It's a joy, let me tell you. Nothing ever gets installed or removed again. Blowing up your system and requiring your OS to be reinstalled is ridiculous. Especially since that was a problem that didn't even used to exist. OSs were once stored on roms.

> /opt isn't new. Plenty of people decided that they could throw everything into a folder under /opt, including copies of all dependencies, with no references outside of that folder, and that would be enough.

Sure, as long as you never move it from opt. UNIX devs' addiction to hard-coding fixed paths is difficult to break.

> And it might have been enough if it wasn't for sysadmins in charge of uptime tearing their beards out over /opt constantly filling up and causing multi-product outages when it did.

Is that really a problem outside of Linux, which famously defined its base-system as "the kernel" and therefore necessitated that any such install would need to bring along all its own dependencies because it couldn't rely on anything existing except the kernel?

> Of course, today, with the price of resources having crashed through the floor, and containerization having more or less reached maturity, this is all more or less an academic, historical, moot point. But dynamically linked libraries weren't stupid, they were an artifact of an age where statically linking libraries was horrendously uneconomical and the idea of namespacing anything was enough to get laughed out of the room by people wondering how you would expect to pay for any of it.

Yet we still live with this bullshit today anyway and the Linux community especially eschews any attempt to move on. This despite some of the same people who created UNIX creating Plan 9, which rejected dynamically linking and embraced namespacing over twenty years ago!

Fedora Silverblue aspires to implement read-only root for a general-purpose distro. The main cost of their implementation is that you have to reboot after installing or upgrading packages to make the changes effective.

ZFS also protects you from data corruption, while mere backups don't - they'll happily overwrite good files with corrupt ones.

Not if you are snapshotting, this is what I do. Rsync makes this very easy and space saving.

My main threat scenarios are hardware failures and encrypting trojans. Snapshotting (if done right) protects also against the latter.

Will you detect data corruption before your snapshots rotate? ZFS controls saved data. Also, it saves more more space then rsync as it can store just the difference (rsync will copy whole file if changed, which makes things like snapshots of virtual machine very impractical).

ZFS root is cool because it allows using snapshots for the entire system. Failed upgrade? No probs -- just revert to a previous snapshot!

> Will anyone, and if so for what purpose?

Boot environments are very handy for updates:

* https://ramsdenj.com/2018/05/29/zedenv-zfs-boot-environment-...

* https://mwl.io/archives/2363

Also, being able to snapshot the entire system at regular intervals, show diffs between snapshots, and then perhaps do security analysis on the results might be handy.

I use ZFS on root on Ubuntu for its flexibility and relative ease of use. It’s easy for example to set different compression ratios on various paths by creating arbitrary volumes. Plus the checksum protection is nice. You can use snapshots for backups too.

> I just don't see myself using it. Will anyone, and if so for what purpose?

probably everyone if/when ubuntu decides to make this the default, which they probably will, because `sudo apt install coolpoints`

I had multiple cases when a system upgrade broke my system but a snapshot saved my ass on my home server.

Reply on news.ycombinator.com