NixOS: Declarative Builds and Deployments
nixos.org
nixos.org
I recently switched to NixOS because I wanted what they were selling and the experience this time around was way better. Having no other option but to figure it out made me learn the essentials real quick (like an exchange program to a foreign country that speaks another language).
If you think about it, when you used Ubuntu or Fedora or RHEL for the first time, and probably for a very long time, you could get by without learning the deep intricacies of what is going on behind the scenes. The same is true with NixOS. The things you need to learn are different, but once you get a basic setup with home-manager setup you're off to the races. (Btw, I used this "book" to get started and it was great: https://nixos-and-flakes.thiscute.world/)
The best part about using NixOS so far is that things just work. Setting up my graphics card was as simple setting enabled = true. Same for configuring specific audio frameworks. And I had tried many times to get Davinci Resolve working on other distros and always encountered issues leading me to need to dual-boot Windows so I could do video editing. Now I just enabled Davinci Resolve and it works! No more Windows.
If you're brand new to linux on the desktop, I wouldn't recommend it. But if you've been doing that for years, maybe try NixOS in 2024.
Another neat trick is reproducible shell scripts that use Nix to declare their dependencies in a sort of extended shebang line.
1. It tries to replace ALL package managers due its extreme philosophy. I had a lot of issues when it tries to replace Cargo for example
2. It uses a functional language, which will lose 95% of users
3. It never had a great user experience especially if you’re new and trying to learn or if you’re not using linux or if you bring your own tools (e.g. vscode)
I think something like that could totally exist one day, but it’ll have to restart from scratch on a cleaner base IMO
That being said, it's really not that difficult to use tools in their original form on Nix is you want. You can create temporary FHS shells in which tools just work. I do this with Python, as I can't be bothered to do it the "right" way when I just want to run some random script from github.
My interpretation would be something like: the abandonment of software that is so poorly designed that it is difficult to package and/or run under Nix.
This commit message (from one of my commits) details some of the struggles supporting Ruby under Nix:
https://github.com/NixOS/nixpkgs/commit/b6c06e216bb3bface40e...
Each of those problems is due to either:
1. Some unmotivated contrivance in Bundler, where the maintainers refused to make their stuff less needlessly broken, or
2. Ruby programmers in general not programming with packaging in mind (haven't touched Ruby/Rails professionally in a while, but when I did, it was par for the course to rsync/capistrano files around -- no one saw the utility of any sort of packaging)
And the two really reinforce each other. Bundler is the de facto way to declare and pin dependencies at the app level, but then Bundler makes it nearly impossible (see the commit message for details) to package software using Bundler, which reinforces the "fuck it, we'll just rsync files around over SSH", which means no one pressures Bundler to Do The Right Thing.
It's the same thing everywhere else. There are complaints elsewhere in this comment section about the nodejs/npm experience on Nix: same underlying problem. The design behind npm is so unnecessarily shit-tacular that it kinda sorta just barely works on its tier 1 platforms. I don't envy the brave souls that have worked on supporting npm packages on Nix.
Guix and Hurd and whatnot try to be more pure about things and look how far it got them. There are tens of thousands of projects that would basically need to be redesigned and written from the ground up to achieve what you desire, and I think the best approach is to build the platform and prove its value and create a path and model for new projects to follow. There are only so many open source developers out there.
And I just realized: I should probably clarify what I meant when I suggested "the abandonment of software that is so poorly designed that it is difficult to package and/or run under Nix" as the means for a "cleaner base".
While that would make things cleaner in some sense, it's not what I'm suggesting Nix should do. I was mostly trying to clarify that the majority of what needs cleaning is extrinsic, rather than some intrinsic failing of Nix itself (though things could certainly be improved there, too).
The greatest source of difficulty for packaging/running things under Nix (having contributed at least a couple dozen packages, and the entirety of the Ruby/Bundler support) is software violating well established principles. To pick one silly example: say you need to get the home directory for the current user. Yes, you could do the equivalent of "/home/${USER}", and while that might coincidentally work on 99.999% systems, that's still wrong -- there's nothing keeping someone from having their home dir be something arbitrary like "/home_is_where_the_heart_is", and now your software will break. The funny thing is that the right way to handle this is also the easy way: use "${HOME}" if it exists, and failing that, fall back to `getpwuid` (or whatever your language calls it). It's easier/simpler because you no longer need to make any assumptions about how the file system is laid out -- either the user tells you through $HOME or you consult the password entry. Easy. But what happens when you send a pull request to fix the objectively wrong logic of the first example? You get push back saying "well, it works on my machine, and this is the first complaint we've heard, so no, I don't think I'll merge your request".
So, I guess I'm being kind of roundabout, but the takeaway I'd like for everyone to have is this: if working with your programming language is difficult on Nix, please go look for (and upvote) open issues in your language's issue tracker (they're probably there) from other Nix maintainers begging for fixes that would not only make it easier for Nix users, but would also remove latent correctness problems that have gone unnoticed solely due to the coincidental uniformity of all other commonly used distros. Just because something works everywhere other than Nix doesn't mean that the code in question isn't needlessly complicated/fragile and/or dependent on non-standardized assumptions that just so happen to be common on other systems.
I’ve encountered NixOS through Replit and thought it was such a fantastic way to manage environments, when I’m using things like ASDF and Bash scripts for installs, I’d love this.
Then comes the question “I won’t commit to something like this and I don’t know why”, and you just articulated it for me.
Nix solves real problems that Cargo can't/won't. And the 95% use case worth optimizing for isn't someone authoring flakes, but people just looking for the proper incantation to get the software they need for their environment.
So it’s more a question of whether it’s a nicer experience to wrap up a pure core in some impure outer layer, or just make the whole thing pure.
Nix the language is imo pretty nice for what it does, and lots of other config languages seem to have ended up in similar kind of “json with functions” part of the design space (e.g. dhall, jsonnet). My personal view is that purity in the nix language helps more than it hurts, and that the big design fuckup wasn’t writing a custom DSL, but writing a custom DSL without static types.
ISTM that if you dislike that, then there's GUIX.
Very briefly, AFAICT, it's "Nix but using Scheme".
- Bazel wants to cache remote resources, like each respective crate's source files.
- Bazel then wants to build each crate in a sandbox, and cache the build artifacts. Because bazel owns this process, instead of Cargo, it can trivially distribute those build artifacts to other developers, or in CI, etc -- instead of everyone having to build everything from scratch.
This is an established practice, and Nix wants to drive the build for the same reasons.
See:
For the functional language and UX, I agree there's room for improvement. If you don't want to dive into learning Nix, you could use a wrapper that simplifies the UX for creating projects. We built one called Devbox (https://jetpack.io/devbox), which uses JSON for it's config format and has a CLI inspired by more familiar tools like Yarn.
I contributed to the community for about a year and have since left for a number of reasons, but this is near the top.
I allocated a Debian and ran nixos-infect on it and started from scratch, even though I have done this for two other machines, because I’m still experimenting with the basic setup.
I do commit changes to git and fork it on company git. But I don’t redeploy remotely: https://discourse.nixos.org/t/deploy-nixos-configurations-on...
I’ll need to make noVNC a part of the deployment (not just its dependencies, like python3).
I should deploy using AWS AMIs, and make a config for the dashboard.
But the NixOS on Hetzner docs are not nearly as easy to guess the best path through. Can you recommend a way to boot NixOS on a Hetzner VPS that is better than nixos-infect?
https://nixos.wiki/wiki/Install_NixOS_on_Amazon_EC2
http://jackkelly.name/blog/archives/2020/08/30/building_and_...
Cloud-init can be clumsy to get going, but it’s possible.
1. Boot with hetzner recovery OS
2. Install Nix in it (not nixos!)
3. Install nixos-generators
4. Generate kexec-bundle image
5. run the result of 4 and continue installation from it.
[1]: https://www.youtube.com/watch?v=wr22CyoyRo4
Apologies if there is already something like this in work by the core nix devs.
I'm not a fan of nix but I'd say the nixos repository is fairly mature.
source: https://repology.org/
Alpine also sounds like a nice system but I'd bet it has an even worse issue with bad packages because of musl.
...I sometimes wonder how the world would be if the "explicitly installed packages" functionality was better exposed and advertised. I never see any guide for "installing development dependencies" use things like "pacman -S --asdeps" so you can clean them up later.
Still not as good as something like npm's `package.json` where it's a human edited list of packages you want installed and it's in the order that you added them in (if you ran `npm install $package-name`). Which is nice when you want to remove one you don't need anymore.
I wonder if it's possible hack a declarative package manger into a Debian system. Like a wrapper around apt that can do all of this.
1. While the WHAT of NixOS was obvious from the start, I have never had a more difficult time understanding HOW to get there with a piece of technology. The documentation is hilariously inconsistent.
2. I have also never encountered a more satisfying, useful, and time-saving piece of technology. I had a computer crash and it took me 15 minutes to be back to where I was on a completely different machine with about 2 commands.
3. Time-saving other than the vast learning curve anyway.
4. Unlike some tech projects, it’s not hard because it’s incomplete or poorly structured. It’s hard because it’s doing something different. (It’s arguable how different it is in 2024, but that’s a different conversation.)
5. To create my child’s account, I spent about 10 minutes editing a copy of my config to pick and choose the packages he needs, and it deployed (on the first go) in about 10 minutes, fully configured.
Projects like fleek that put a simplified face on nix are incredibly valuable, because they can unlock 65% of the benefit of the platform with almost zero effort. I can’t tell you if the time investment for the extra 35% is worth it for someone managing family computers, but, y’all, I spend less time fiddling with them now and they break less often. That’s a … plus?
If you're interested in trying out NixOS I highly recommend either running it as an WSL2 VM[1] if you have a Windows machine or deploying it on a cheap VPS[2] to play around with.
I personally still use a Win11 desktop for compat with Adobe software and a few other bits and pieces, but I do all my serious work on a NixOS WSL2 VM.
To me, I tried Nix, got frustrated with packages, realized I'm just using someone else's abstractions for defining configuration files (with the same kind of quality you'd find on Ansible galaxy), and realized it's not solving a problem I have.
"How we went from supporting 50 languages to all of them"
Then looked at NixOS. I geht the point of having a deklarative like OS which you can rebuild in a couple of minutes, BUT the nix language is kind of unituitive and all the tools like flake and nix packages do some magic it con't quiet understand. It does not help that the docs are kind of meh.
In my experience, they’ve improved massively over the last few years.
Most of the time, the docs directly explain how to solve some problem I have while also giving good context. I recently was messing with my boot loader and the docs were perfect
Sounds like podman. What is the big differentiator from it?
Nope.
At best, you can do something like this: apt-get install gparted=0.16.1-1
That handles the semver case, but that doesn't address the rest of the "all of the above":
- sha256 calculated by the content of the package (Content Addressed)
- checksum calculated from all the inputs that were used to create the package install spec (Input Addressed)
> Nix isn’t the first package manager to do something unique.
Sure. But it is the first package manager to do the above unique things. After all, the reason for Nix's existence is just that: no other package manager before it has done what it does.
You need the following software:
- You need the latest release of Erlang for your app
- A database (like Riak), whicn in turn happens to need an older version of Erlang
- The new Erlang needs the latest libopenssl
- The old Erlang needs an old v1.x libopenssl
- You used some open source software written in C++, so you need libstdc++
- Some of your own C++ code has exposed a bug in libstdc++, so you need yet another libstdc++ with your patches, but just for that software
- Etc
How does apt-get install allow you to have multiple versions of a given dynamic library (with the same SONAME) and multiple binaries (erl, clang, etc) installed at the same time?
The answer: it doesn't.
Nix has no problem handling everything I described above, and it handles it easily.
So, yes, you can pin versions with apt-get install, but you're out of luck if any of those packages have any transitive dependencies with different versions.
Also, the pinning guarantees are higher with nix: you can be guaranteed that your packages and their transitive dependencies are byte-for-byte identical, every time. Pinning end-to-end across all packages (and every transitive dependency thereof) is not a normal, happy-path thing to do in apt, so you would be hard pressed to find anyone that does that, given how onerous that would be -- whereas it's trivial in Nix.
Whereas Nix keeps binaries around longer, but also includes the ability to automatically and reproducibly build from source.
More appropriate comparisons would be: - NixOS and Ubuntu - nix and apt/homebrew
No need to use containers too, which bring cognitive/workflow overhead.
If you're asking if installing/running software with Nix requires containers like Docker does: no, it does not.
> At least with Docker you can get something useful within 2 minutes
% nix shell 'nixpkgs#ruby' -c irb
irb(main):001:0> puts "Hello, world!"
Hello, world!
=> nil
irb(main):002:0>
If it takes 2 whole minutes to use Docker, Docker must be pretty bad. I guess vintage things have their appeal, but apart from that I don't see why people would be so attached to such archaic, inferior technology.
Every time I try NixOS on a desktop with anything more complex than setting a couple flags (writing some derivations to work around the broken software, managing my dotfiles etc), my configuration ends up as an unholy mix of weird Nix trickery, FHS paths in a non-FHS environment, and foreign config includes. It's like writing (programming) your Emacs config by patching the existing one with a Python script, for twice the fun. Of course it also needs to be versioned and documented to keep track of it, otherwise what's the point.
I can't help but think that the Arch approach (make the system management transparent, close and personal, and ignore the burning trashcan) is infinitely easier for personal usage than the Nix approach (describe everything with an additional layer of indirection). It's probably the opposite for huge deployments, of course.
I assumed I was just a dummy for not getting it to work because there are tons of peoples' flakes on Github with their entire desktop environment declared. That's what gave me the impression that this was totally normal. It's super cool and impressive but I just haven't grokked reading Nix yet.
You know how a Docker file barely has any structure, leading to a lot of bashisms to try and script certain behaviors? Nix doesn't have that issue.
NixOS is an OS you can configure with Nix. A lot less "OK so edit these files to try and configure something, then reboot", a lot more "configure this data structure, then reboot". Makes your system configuration mostly be in the same place, as NixOS will distribute these changes across the file system as needed to get things working. It's pretty nice!
Overall... it ends up being a bit easier to document why you're doing something in a Nix file relative to a Dockerfile, and caching and the like works much more smartly (lots less "move all this stuff to the top/bottom for caching reasons" you might do in a Dockerfile)
("Nix" here means "nix + nixpkgs", which is important since lots of nix-ism are basically downstream of how nixpkgs is written and the patterns used there)
But I'm yet to use NixOS, so i don't know what it's downsides are..
We open sourced our Nix dev environment tool (Devbox), and wrote a bit about why we switched here: https://www.jetpack.io/blog/devbox-turn-a-1000-container-scr...