NixOS: A Personal Post-Mortem

I think the Nix language is a big barrier to entry for NixOS, I’ve used it quite a bit and still struggle to debug problems, I think lack of types or some basic type inference doesn’t help.

That said I’ve had a very positive experience packaging NodeJS apps with Nix, the /nix/store lends itself very nicely to the mess that is node_modules as each NPM package can be stored once in the store and NodeJS can read the dependencies directly from /nix/store if using something like Yarn PNP, I wrote a Yarn plugin to make this a bit easier https://github.com/madjam002/yarnpnp2nix Using Nix as a development shell along with direnv makes for a very nice developer experience.

The thing is though, you can get these benefits even without using NixOS, you can install Nix on various Linux distros and personally I use it on macOS.

NixOS is great for servers where reproducibility and consistency is important, personally I was never a fan of tools like Ansible that are a bit of a hack.

I've never used NixOS, so I have no horse in this race, but I'm not sure this write-up is entirely fair.

The first issue you describe is that you already have alternative Nix-like solutions in particular domains (R packages), and Nix let you continue using them. Is this not desirable though? You have the choice of letting Nix handle this, or carving out a Nix-less niche to continue using existing / more cross-compatible solutions.

The second issue is that support for games is weak in NixOS. But is this surprising? You're running games via emulation on a niche OS, where neither the game or the emulation tools you're using officially support NixOS. I'd say the fact NixOS can run these games at all is very impressive. Other alt-OSes, like Haiku, would not be able to do so.

The third issue is that you seem to need to edit /etc/hosts to be able to connect to captive portals on public WiFi. Do you? I'm on Linux, and I don't.

It honestly seems like Nix requires more tinkering than you were willing to do to get it set up, and that's an absolutely valid decision to make - it's the reason I don't use Arch, for example - but I feel like your article frames this as an issue with Nix, which seems a little disingenuous.

It's sounds like one of the major problems here was using configuration.nix. Nix flakes provide a much better user experience, but they are currently "experimental" (seriously, its time to stabilize flakes) and you are not guided to use them at all.

I have .envrc ignored in my global gitignore, and then I write 'use flake $HOME/my-nix-config#node' to .envrc. direnv then grabs that and does the rest. I no longer need to commit anything Nix related to git, so my coworkers are none the wiser.

Ultimately this article shows that, once again, the problem with Nix is the documentation.

    What you might not be as familiar with is that these authentication sites often block you from visiting any domain at all, and then to access the authentication site on their own domain you must explicitly link their domain to an IP address through the /etc/hosts file.³
    
    3. I'm assuming on Mac and Windows this is taken care of for you somehow and it is a non-problem.

I... have never had to do this in any distro or OS (NixOS, Arch, macOS at least). The way I've found captive portals to work in practice is that when I connect to the WiFi, my computer gets the captive portal's DNS server through DHCP, and their DNS server points any and every website to the IP address of the captive portal. To make it even easier, Firefox checks "detectportal.firefox.com" every so often to see whether or not there's a captive portal, and gives me a prompt to log in to it if need be.

    To edit it, you must edit your system Nix configuration in /etc/nixos/configuration.nix and then rebuild your system, which —get this — requires a network connection to complete! So somehow, to connect to the Internet, you must be able to connect to the Internet.

I'll be pedantic and say that this isn't true, it's not required for you to be connected to the internet to rebuild your system when you're only changing /etc/hosts. However, the fact that it's not immediately obvious _how_ to do this is a rather bad problem with NixOS's tooling.

For what it's worth, the following worked for me, though there appear to be other ways as well:

    cd /etc/nixos # Or wherever you store your configuration.
    nixos-rebuild build
    sudo nix-env -p /nix/var/nix/profiles/system/ --set ./result
    sudo ./result/bin/switch-to-configuration switch

And finally, for a neat report from a few years ago on using NixOS without an internet connection: https://discourse.nixos.org/t/using-nixos-in-an-isolated-env...

I keep having to say this to people - if you're unsure about nix/nixos, "diving in the deep end" and using nixos as your desktop operating system is probably just going to annoy you. I really suggest using nix on a regular distribution for a good while beforehand.

Of course, IMHO, servers is where nixos really shines.

I find the nix language beautiful.

The vast majority of the issues I have faced have been down to nixpkgs. Of course, nixpkgs is an incredible project, but it lacks direction, uses bad practices everywhere, and pretty much all of the flaws you’d expect on a vast monorepo with a somewhat laissez-faire management approach.

Oh, one other problem I had, I guess, was when MS removed a link to a Teams binary that was baked into nixpkgs, which meant I couldn’t rebuild my system without losing teams, until I fixed it with a custom derivation. I could also have fixed it by pinning the nixpkgs it came from to one with a working teams URL, but that seemed like too big of sledgehammer for a single application.

One of the key assumptions made by nixpkgs, and by many hermeticity-focused projects, is that URLs stay alive and unmodified. They don’t promise that something will always build. They promise that it will build perfectly, exactly as you specified, or not at all. There’s a nice security element to that I guess.

Nonetheless, NixOS is the best distro I’ve ever used. I certainly spent much more time fighting Arch, etc than I do NixOS - and I’ve never come across a problem I can’t work around with some nix code.

So my position, I guess, is that NixOS is the second best distribution for me. The best is the one that builds on NixOS and has learned some mistakes, thrown away some technical debt, etc.

I've had a rather similar experience with NicOS a few years ago. I did get comfortable with the language and even contributed a small package! I did like having a reproducible config file for my whole OS setup.

However, everything was just a hassle to get working. Any program that's not in the package repos takes a day or two of fiddling to install. I just lost patience with it and have been on Ubuntu for the past few years.

I spent 2 years using NixOS as my sole development OS and I learned a ton. But every little thing is a battle. Prior to that I used Arch Linux for like 8 years and it was so much more pleasant to use for day to day dev (due to the great package manager and high quality packages and, well, mutability).

At some point last year I came to the conclusion that the immutability/repeatability of NixOS just isn't worth the drawbacks for a development machine. I swapped to openSUSE Tumbleweed and haven't looked back. It never gets in the way - I install what I want, manage my dotfiles in chezmoi, and rarely need to fight the OS to do what I need it to do.

I think NixOS has some interesting applications at scale. If I had to manage a fleet of 100+ machines and ensure they are all identical, and I can easily roll back changes, NixOS would be a prime candidate.

Similarly, I still think Nix the package manager is pretty great for individual project dependency management. There isn't really anything like it as far as I know.

We were running nix in production and it was an absolute mess

While the language is declarative your configuration is scattered, debuggers are absent, and you're ultimately interacting with a a layer on top of what the original package was designed to do.

This makes it hard to figure the system out, how to debug, and occasionally support is missing.

I also tried nix for 3-4 weeks only to stop short of going all-in on my desktop and laptop. I really like the concept but I found the nix language to be too difficult to pick up fast enough to be useful to me. I certainly loved being able to describe my system via nix scripts so they are reproducible.

In the end, I realized I was just still using ArchWiki for help and also that my Arch installation on my desktop was still fine after 5 years. I ended up switching back and felt much more at home. Personally, I would consider nix again if the language and scripts had less learning curve (I couldn't get vim code completion to work, ctags doesn't support nix either). Otherwise, I'd love to see a similar distribution in concept that used a more standard language that might be more accessible.

My garbage can is a few meters away from my desk. That's too far -- guess I'll just litter on the floor -- it's more convenient. ... but soon the trash builds up.

Upgrade the whole system? Dangerous. Upgraded and it kinda works, but you'd like to roll-back? Though luck.

NixOS is definitely not for everybody - perhaps Developer workstation is the most tricky to get right. But for production server use, the repeatability is great.

Nix ofc is agnostic of NixOS, is is generally my preferred way of getting packages.

I also happen to love when software makes too-wide assumptions and break. That is generally software poorly written and to avoid.

These side-effects and mutations cross-product with each other so much that computers have become failure dumpsters.

Really well thought out. Only a true hacker would conclude with “and so that’s why I’m going to Arch Linux for the user experience”! :)

It’ll be funny if dependency management turns out to be a categorically harder problem than consciousness.

Thanks for the great write-up. As someone who somewhat recently got into Nix I've struggled with many of the same problems.

With the whole "your system is read-only" bit, I'll add that Nix can make trial and error editing of config files quite tedious. Editing and reapplying the configuration is fast, but not as fast as editing a file and `systemctl restart`ing the service in question.

On development environments I've also had similar problems. My workplace uses a combination of Python and C++. While Python has an established package management system, C++ is still in the early days. The traditional is just use what your OS provides, but the production OS is different from the development OS, the former being much more "stable" (and therefore with older package versions) than the latter. This has caused plenty of headaches with features not being available, and is holding us back from using newer C++ features, or even adding dependencies easily.

Being keen on Nix, I've been testing out using it locally for dependencies.

With C++ it mostly works. Some annoyances have been packages not providing CMake or pkglib config files. Some packages don't quite have all the possible features enabled, and it's not trivial to enable them (I struggled with arrow and liborc).

Then comes Python, which introduces the same nightmares which the author experienced. Python packages that use system shared libraries are troublesome. The buildUserFHS solution seems like the right approach when mixing and matching, but it introduces a few more problems (the details of which elude me right now).

An additional problem is portability. Theoretically one could use `nix bundle` to ship whole applications, but a shared library is a different matter. We've got a Python module written in C++ module, I'm not sure a module built by Nix would be compatible with the host's Python, given the libstdc++ differences and so on.

This struggle is topped with the fact I'm the one going against the grain, trying to sort out the issues I hit, for my own purposes only. I do hope I manage to prove to myself that it's a tenable approach, and eventually can propose it to the wider team. But for now I know it's a dream.

For the record it is not correct that NixOS's choice to store language packages in its own repos is unique: Slackware does this and IIRC Ubuntu and Debian both do this. In fact, this is so common that I occasionally see warnings on package websites telling you not to install it from your distro's repos specifically because it will be out of date.

I really hate Hate HATE nixos. If any good alternative to nixos shows up, I'd use it in a split nanosecond.

Meanwhile, the advantage of reproducible builds and rapid rollback are so huge for me that I use it anyway.

May the source gods have mercy on me. :-/

I prefer Ansible