Vanilla OS 2: an immutable distribution to run all software | Hacker News Reader

Vanilla OS 2: an immutable distribution to run all software

lwn.net

196 pointssigna1151 comments

I like that they went from Ubuntu to Debian as the base OS. I assume they target Debian Sid (the unstable flavor of Debian) because they can compensate for the instability with the immutability provided by ABroot. Or is it simply because the distro is pretty much in a rapid development stage?

The distribution looks very fun. Something quite new to play with for distro-hoppers and to learn more about some techs.

Last time I tried fedora-silverblue I didn't like it. My packages were scattered in 2 or 3 distrobox containers. It's not that much, but they can be different distributions, and then we add flatpak to the mix, and apps installed in the base OS with rpm-ostree... It felt like a frankenstein distro. Upgrades were time consuming, and not smooth at all. Not only did I have to learn how to manage a fedora-silverblue, but I also had to maintain a debian container, upgrade another fedora (a regular one, not silverblue), learn the quirks of flatpak, and... that was too much work. It doesn't really matter that I can confidently upgrade the empty base OS, if I still need to manually upgrade my fedora container and it can break the package I need from that container.

The approach here with Apx is worth a closer look. It abstracts away the different package managers of the main distro (`apx search` PACKAGE will translate to `apt-cache search` in debian container, `pacman -Ss` in arch container, `zypper search` in opensuse...). The concept of "exporting" the packages, and the UI around it, makes me think they aim at making the management of these distrobox containers easier.

> Last time I tried fedora-silverblue I didn't like it. My packages were scattered in 2 or 3 distrobox containers. It's not that much, but they can be different distributions, and then we add flatpak to the mix, and apps installed in the base OS with rpm-ostree... It felt like a frankenstein distro. Upgrades were time consuming, and not smooth at all. Not only did I have to learn how to manage a fedora-silverblue, but I also had to maintain a debian container, upgrade another fedora (a regular one, not silverblue), learn the quirks of flatpak, and... that was too much work. It doesn't really matter that I can confidently upgrade the empty base OS, if I still need to manually upgrade my fedora container and it can break the package I need from that container.

I... don't think you're supposed to do that? Like, I haven't used silverblue specifically, but on suse microos there's a big warning with metaphorical flashing lights that says in so many words "don't touch the host OS unless there is literally no other way to possibly do what you need". You should never use it for normal applications, at the very minimum. And as to multiple distros in multiple containers - why not just run everything on Fedora container(s)?

I'm aware. I think I tried. At the time (silverblue 26) they recommended to use a firefox installed as rpm (actually, the GUI was offering both flatpak and RPM for some packages, and "RPM" meant it used rpm-ostree under the hood to overlay the package). At least they warned about some issues when using firefox as flatpak (mostly related to integration with the rest of the desktop environment). And so, in order to get some things to work I had to install a few packages with rpm-ostree. Digging in my docs, it was:

> # nvidia and codecs necessary for firefox and youtube: > # You'll need rpmfusion repo (see dedicated section for how to install them. Ugly.)) > rpm-ostree install akmod-nvidia ffmpeg xorg-x11-drv-nvidia xorg-x11-drv-nvidia-cuda

Maybe I used it wrong, idk, but here is how my .zshrc looked like:

> alias "hx"="toolbox run -c devops hx" # I installed there all the mess for LSP server > alias "aws"="toolbox run -c devops aws" > alias "mpv"="flatpak run io.mpv.Mpv" > alias "yt-dlp"="toolbox run yt-dlp" # default distrobox is fedora

I could go inside containers ("activate" containers), but then, if I want all my tools... they need to reside in the same container, right?

I didn't feel like installing all the utils in all containers, or running exa and ripgrep in some fedora "basic-utils" containers and adding more aliases for very basic tools. So I ended up overlaying the utils I cannot live without, thinking they are not really unstable software, it can't possibly break the upgrade (and indeed it never did for the time I used silverblue) :

> rpm-ostree install bat exa git-delta ripgrep vim zsh zsh-syntax-highlighting zsh-autosuggestions fzf jq

I also needed some stuff to fix the thumbnails of the default gnome (I don't remember, but I'm pretty sure that if I did it with rpm-ostree it's because I didn't find another way):

> rpm-ostree install ffmpegthumbnailer gstreamer1-libav gstreamer1-plugin-openh264

I also couldn't install some ibus packages (with too much integration with the desktop/keyboard) in a container so I resorted to rpm-ostree there as well.

So while I really tried to keep everything out of rpm-ostree as much as I could, I felt like it was a constant trade-off: going against the spirit of the distribution VS managing every single util and running little cli tool in containers (that need to be maintained).

I'd be happy to read about some workflows, the "correct way to do it", or if silverblue changed since the last time I used it. But for me it's in the design itself: "use containers" mean "do the plumbing between or your tools yourself" (even if distrobox makes it easier by exposing/sharing pretty much the whole home, network, env vars etc...)

That's fair. Certainly I would expect the thumbnail and ibus stuff to need to be on the host - although I'd also argue it was a bug that you needed to bother; that seems like stuff it should handle properly out of the box? I'm pretty sure the flatpak situation has improved with time, though again bear in mind I'm on a slightly different stack. And the general friction of having all your tools inside/outside the containers and needing to glue it all together... I think it's a question of workflow? Like, if I was on silverblue and decided to ex. write or work on some C program, I'd make a container that installed both the specific tools (gcc, make) and my stuff (git, vim, fzf)... or actually I'd probably stick them in 2 containers with a source code directory mounted into both. The funniest thing I hit was needing to make a container to get git when I wanted to create a repo to hold my Dockerfiles first. So it's definitely a bit of friction, but if you're comfortable defining everything you're doing programmatically then it's a bit of pain up front followed by being able to regenerate your entire environment in a few commands: If a new Fedora release drops, you don't upgrade your containers; you bump the base image, build, and cut over to the new container, and if you hit problems you revert to the old tag. If you don't like that approach then yes it might be annoying. Or, if you prefer pets to cattle, you'd need everything you want to be available in one distro so you have to figure out the host/container demarcation but then it's all one container to manage.

I'm on Kinoite. I don't need stability in my dev environment for now. I use a single toolbox based on Fedora rawhide. I get the best of both worlds, immutable distro and streaming distro and it keeps things simple. I agree Silverblue would benefit from an integrated management system / UI.

I think all the instances of people saying they like it, are just examples of being trained and accepting the training, to just not be a tinkerer and just use what's given to you by someone else without touching it.

Not only being unable to customize or hack or basically just use a toolbox rather than a product, but being indoctrinated over time, especially new users, not to even think of it in the first place. Pretty much just like commercial software. Why bother running linux in that case? We already have polished untouchable product software.

When I say "unable" I don't count "able by dint of suffering 1000 paper cuts".

Using package layering (rpm-ostree install [...]) used to be discouraged unless strictly necessary, but it's no longer the case: https://docs.fedoraproject.org/en-US/fedora-silverblue/getti... . Nowadays you can as well use only that if it suits you.

One of the Vanilla OS lead developers Mirko also made Bottles[0], a lovely piece of software for running Windows apps on Linux without fussing about the command line

[0] https://usebottles.com/

Happy with Fedora Silverblue in that space: https://fedoraproject.org/atomic-desktops/silverblue/

Especially Project Bluefin: https://projectbluefin.io/

Of course, it's great to have a Debian based alternative!

silverblue is fucking awesome (coreos rules)

i'm kinda surprised this project didn't use ostree with apt.

Ostree might be replaced by bootc eventually. According to a talk at Flock this year.

I'm glad Debian didn't use ostree because it's nice to have more than one implementation of immutable container based distros.

that's cool to know. I've not heard of bootc before!

+1 for Fedora Silverblue and Bluefin.

Recently made the switch after having a terrible upgrade to Ubuntu 24.04, very happy so far with the use of brew and flatpaks in the OS.

I’m using Bluefin as a mostly daily driver, and it has some rough edges, but it’s mostly OK (and my distrobox is Ubuntu, so most of my dev needs are covered).

In between Bluefin and Bazzite, most of my zero-maintenance Linux usage is catered for (although I still rely mostly on macOS).

> Orchid ensures your system is always up-to-date without interrupting your workflow. With smart updates that check only when your device is idle, you get the latest features

The screenshot doesn't seem to show a toggle to turn on/off the automatic updates, not sure if that's because you can't, or the screenshot is just a mock up?

I'm probably not alone in that I stop updating my OS and related software while working on things with deadlines, as the risk of breakage tends to be non-zero, and surely a new OS today has to at least allow people the choice for automatic updates?

You are not alone. I've been using immutable Fedora for the past 2 years and obviously I want control over when it updates.

Even if it automatically installs an update, it doesn't take effect until you reboot, right? And even if it does break something you just reboot and select the other partition and that rolls you back I think?

(Although I agree that there should still always be user control)

From the linked article:

"Vanilla OS also checks for system updates weekly, by default. Users can change this to daily, weekly, or never if they choose. To minimize any impact on the user experience, the updater considers factors such as system load, battery level, and network connection before applying system updates. With the core operating system and applications set for automatic updates, Vanilla OS aims to relieve users of the often tedious task of keeping all packages updated."

As long as the update process actually works, I think that's a good balance - weekly by default, but can be disabled.

I was expecting something like Gobo Linux, with various package managers installing packages into their own non-conflicting paths with Gobo symlink wizardry, but instead:

> [...] through Apx, a wrapper around Distrobox. The latter, in turn, is a wrapper around Podman, Docker, or the simple container manager lilipod

I’m glad to see the author note that the installer is problematic - I think I’ve only had 1 seamless install in about a dozen attempts.

I filed a bug report to add a function to copy error messages from the installer - would be really helpful to file meaningful bugs reports

"Oh the kids these days... they don't even know the correct name is OS/2."

(loads article page)

"...Whooops!"

Interesting that they're going with full A/B partitions; it's a touch simpler and more resilient (ex. against disk corruption), but ex. ostree is more space efficient.

I'm also curious if there is really no way to install things on the host - is there a way to install NVIDIA drivers or the like?

It looks like you can do it that way according to the docs [0] :

> abroot pkg add PACKAGE_NAME

Not sure what it does exactly under the hood. I'm not sure it persists after an upgrade.

In the release announcement blog post they also mention a way to build your own (base OS?) image with something called VIB [1]

Regarding space efficiency, the distribution relies on a "LVM Thin provisioning" feature. [2] I don't know enough about it (nor about ostree) to compare the two.

This is all very refreshing! Many new techs and concepts to look into :)

[0] https://docs.vanillaos.org/handbook/en/install-additional-dr... [1] https://vanillaos.org/blog/article/2024-07-28/vanilla-os-2-o... (section "Make it Truly Yours") [2] https://github.com/Vanilla-OS/ABRoot#thin-provisioning

Most modern Debian/Ubuntu installations support overlayfs :

sudo apt install overlayroot

#edit the setup script vars

sudo nano /etc/overlayroot.conf

overlayroot="tmpfs:swap=1,recurse=0"

#set different role contexts for grub

sudo nano /etc/default/grub

GRUB_SAVEDEFAULT=true

GRUB_DEFAULT=saved

GRUB_TIMEOUT=3

GRUB_RECORDFAIL_TIMEOUT=$GRUB_TIMEOUT

GRUB_TIMEOUT_STYLE=menu

GRUB_TERMINAL=console

GRUB_CMDLINE_LINUX_READONLY="quiet splash i915.tuxedo_disable_psr2=1 i915.enable_psr=0 "

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.tuxedo_disable_psr2=1 i915.enable_psr=0 overlayroot=disabled fsck.mode=force fsck.repair=yes "

#etc...

#then insert an auto menu item for the read only OS boot up

sudo nano /etc/grub.d/10_linux

  if [ "x$is_top_level" = xtrue ] && [ "x${GRUB_DISABLE_SUBMENU}" != xtrue ];

then

    linux_entry "${OS}" "${version}" simple \
    "${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_DEFAULT}"

    linux_entry "${OS} READ ONLY OS" "${version}" simple \
    "${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_READONLY}"

#etc...

#and finally enable the role selection with your current kernel

sudo update-grub

sudo update-initramfs -u

For systems that have enough ram and cheap SSD it ensures the flash memory will last longer. Additionally, running /home on F2FS for workstations can improve long-term hardware health, as Desktop users do not require the OS to be writable in many use-cases.

It is not a perfect approach, as silly things done as root can always bork the backing partition.

Best of luck =3

Yes, quite trivially...

Docker "/var/lib/docker" images may be placed on a mounted mutable partition path:

https://www.digitalocean.com/community/questions/how-to-move...

Hardly the epic task it once was... =3

I'm referring to this https://garden.glennstovall.com/notes/build%20your%20own%20d...

Indeed, some products gain traction for unfathomable reasons:

https://en.wikipedia.org/wiki/Pet_Rock

Have a great day =3

You too!

Do you happen to know if the overlayfs bug that upset the firejail developers has been fixed yet? https://github.com/netblue30/firejail/discussions/4178

Perhaps firejail is dead? There's been no releases in 18 months.

Personally, most projects outside the kernel are not really a priority for monitoring. The packaging ecosystem has always been a bit messy, but I would recommend sending the Debian and or Canonical admins a request to revoke the developer signing key to purge the problem/abandoned firejail application package moving forwards.

Best regards, =3

This looks like an enterprise solution, because why would you install such a locked-down OS on your personal computers? Experienced users don't usually screw up their systems to need this thing, and new users don't touch the dangerous settings in the first place.

But the website makes no effort to relate to enterprise-level administration. Strange.

> Experienced users don't usually screw up their systems to need this thing

It’s less about experienced users screwing up and more about safely running software.

I do lots of small experiments and regularly clone repos from GitHub to try various projects. I prefer to do this in an environment that’s been a bit more hardened and locked down. I like the ability to get my system back to a known good state.

I’ve primarily been using NixOS as a daily driver, but have been pretty curious about Vanilla OS and plan to try it out.

Which, again, is largely an enterprise/professional use case, don't you think? Why is this being marketed as a general-purpose OS then?

> I do lots of small experiments and regularly clone repos from GitHub to try various projects

This is something done often by many in their free time for personal reasons.

I do the same thing all the time!

It has become kind of a superpower being able to deal with many different build systems, general nuisances trying to run both old and new software and just being in the know about the capabilities of all kinds of software.

It's fun and keeps the blade sharp.

Security isn't an enterprise only feature.

I’m currently on sabbatical. No enterprises involved.

Good security hygiene and having a flexible but safe development environment are not limited to the enterprise, and are practices that I apply to every project regardless of whether it’s personal or professional work; a distinction that doesn’t really make sense when you’re working with untrusted software.

For an experienced user, the value proposition is efficiency. Sure, I can spend 3 minutes thinking about how not to screw up my system / my existing hobby projects every time I update a library. But if I'm using something like Vanilla, I can spend all those 3 minute chunks of time on something else.

Which part do you consider locked down: a "sealed" root volume, or apps purely running in a containerized/sandboxed environment?

the second one.

I've been using immutable Linux on both my workstation and my servers for 2 years now.

It definitely fits into the enterprise niche, but imho it's good for everyone. Mainly because of the ability to quicly revert back to a working state. One of the worst things that can happen as a Linux user is that some update causes an issue and forces you to spend the rest of your day troubleshooting.

Since switching to ostree I just revert back to the previous working image, pin it, and continue on with my day. Trying the update later to see if the issue is resolved.

This out of box is incredibly useful to beginners who don't want to learn how to troubleshoot Linux. But also enterprise sector.

In the Enterprise sector I think rather immutable server OS is causing a shift in the perspective of Linux to more of an Appliance than a full blown server.

My last project was actually setting up a series of container hosts on-premises in a Hypervisor. I opted to use CoreOS and could simply disable SSH on the nodes because there was no need to login to any of these nodes. If we want to make an update we re-image them in the hypervisor and re-provision the new image.

This shaves off a lot of issues like for example manual changes, configuration drift, security issues with providing shells and multi-user login.

I use NixOS on many of my machines, because it's also able to easily revert/roll back changes, plus you get declarative configuration and a bunch of other stuff that immutable, but imperative, distros don't give you.

This kind of system is ideal for developers since you can have independent environments for different use cases, even with different distros.

And when you upgrade your base system, they aren't affected. When you have lots of things to manage, this is a big win.

How does it compare to Qubes?

Different goals. Qubes cares very deeply about security. Qubes uses virtualisation to isolate different parts of the system from each other. Immutable distros like Vanilla OS care about providing a reliable base system that can be updated without any problems. They want to achieve this by having fewer moving parts by using readonly container images for all the system software. The user then installs additional user-facing software in their home directory (usually as flatpaks). The additional security is more of a by-product than a central design aspect as it is with Qubes.

This sounds very similar to Fedora Silverblue- immutable OS (rpm-ostree) with rich container support. It uses Debian instead of Fedora of course. Are there other differences though?

Read this as "Vanilla OS2" for a moment.

Has anyone put OS2 in Docker yet?

I don't think that's how containerisation works. It uses the host machine's kernel, so unless Docker has been ported to OS2, there's no way to run OS2 inside Docker.

Vanilla OS is crap. I wanted to try in in a VM and it wouldn't even install. First, it complained that the disk size wasn't enough. Then it complained that it absolutely required UEFI. Then the installer just failed with a cryptic error showing a line of Golang code.

I will try it. Immutability is a great trait. But hey, Linux Mint already is in the category of just works.

Reply on news.ycombinator.com