Immutable OS are great when you have a team of people to manage the tooling and process complexity of deploying and managing them, end users who only want to use software running on those servers, not the servers themselves, and you are bringing servers up and down all the time, then you actually reap the benefits of standardization and infrastructure as code.
But when it's just you, you want to interact directly with the OS, and it's just one device, it's just foot guns all day.
In theory, a generic existing machine could have been compromised by malware, in which case the configuration may not match the previously provisioned version.
With OS launch integrity to guarantee absence of tampering, and prove that current=expected config+binaries, it could be feasible to rerun provisioning config.
# Setup
Setup was a process, no clicking through a nice UI for this one. I had to set up a web server on a second machine to serve the ignition yaml to the primary machine.
It was a very manual process despite CoreOS's promise of automation. There were many issues like https://github.com/coreos/fedora-coreos-tracker/issues/155 where the things I wanted to configure were just not configurable. I had some well-rehearsed post-setup steps to rename the default user from "core" to my name, set keyboard layout, move system dirs to a set of manually-created btrfs subvolumes, etc.
# Usage
The desktop and GUI worked flawlessly. All I had to do was install i3 and lightdm via rpm-ostree. Zero issues, including light 2D gaming like Terraria.
Audio was a pain. My speakers are fine. My mic worked out of the box in ALSA, but Pipewire didn't detect it for some reason, so I had to write some manual pipewire config to add it manually. Also, I had to learn what ALSA and Pipewire are...
I ran just about everything, including GUI apps, in distrobox/arch containers. This was very nice: Arch breaks itself during updates somewhat often and when that happens I can just blow the container away and install pkglist.txt and be back in 5 minutes. I get the benefits of Arch (super fast updates) without the downsides (upgrade brittleness). I plan on keeping distrobox even once I leave.
# Updates
I disabled Zincati (the unattended update service) and instead I ran `rpm-ostree upgrade` before my weekly reboots.
This is the reason I'm leaving. This was supposed to be the smoothest part of CoreOS, but those upgrades failed several times in the past year. To CoreOS's credit my system was never unbootable, but when the upgrades failed I had to do surgery using the unfamiliar rpm-ostree and its lower level ostree to get the system updating again. As of now it's broken yet again and I'm falling behind on updates. I could solve this, I've done it before! But I've had enough. I'm shuffling files to my NAS right now and preparing to hop distros. If anyone wants to try to sell me on NixOS, now's the time ;)