Comment by turboponyy | Hacker News Reader

Comment by turboponyy | Hacker News Reader

NixOS is not for the impatient. Maintaining your system configuration is an exercise in software development like any other - if you lack rigor or discipline, your codebase will become painful to work with. Nothing is ever one shell incantation away - it has to be carefully architected into the existing system.

That being said, once you get the hang of things, you reap amazing benefits:

- You can clone your system to any machine, and immediately have an identical environment

- You can share system configurations as code (declare the means for hosting a website in its repository, for example)

- You can use a fully-fledged programming language to configure any part of your system

- You can make use of an extensive ecosystem of easily composable, prebuilt NixOS modules

- You can seamlessly integrate with Nix, allowing for ephemeral development environments and shells with packages, eliminating much of the need for imperative package management

- Everything in a Nix-based system must be derived strictly from (lockfiled) inputs, making the reproducibility guarantees incredibly strong (barring any network errors or resources being taken down)

- The declarative nature of anything Nix-based means that every change is documented - your system never shifts from the source of truth, compared to other distros where discipline is required to maintain reproducibility

- Nix is so robust that you could even nuke your filesystems on every log out, if you'd like

I can do (almost) all of these things with a traditional Linux distribution, git, and a few scripts written in my language of choice, and I can do it faster.

Nix seems more competitive with something like Conda or Spack than a traditional package manager like Apt or Pacman. Similarly, NixOS seems more suitable for deployment in containerized environments built on woefully outdated deps than personal dev setups with generally higher security requirements.

I've experimented with both Nix and NixOS, and while I adore both the beautifully functional language and the endlessly helpful community, I haven't been able to justify the overhead of maintaining a nix.conf, flakes or no flakes, on top of everything else I need to do on my machines. I kept running into issues where some program I needed wasn't already integrated with the Nix way of doing things or something that worked on my Linux setup failed to work on my macOS setup despite multiplatform guarantees.

Nevertheless, I wait with baited breath for the day that Nix replaces Conda among a critical mass of practitioners in my field. I've tried getting a few of my teams to switch (with help from Devbox and Devenv), but it's a struggle to get people to learn new things when their (often worse) way of doing things works well enough most of the time. Heck, I count myself lucky for getting some people to switch from the atrociously slow Conda to the significantly faster Mamba as even that was inordinately difficult despite for latter being a drop-in replacement for the former.

>I can do (almost) all of these things with a traditional Linux distribution, git, and a few scripts written in my language of choice, and I can do it faster.

As someone who moved from a directory of numbered bash scripts and GNU Stow to Nix, I can't disagree more. The fundamental difference is one is imperative and the other is declarative (and reproducible), and that makes all the difference.

Exactly. I have used Linux since 1994, from Slackware to RedHat to Debian to Gentoo to Ubuntu to Arch.

NixOS and nix-darwin both using nixpkgs is the first time I have the exact same shell and programming/CLI environment on all five of my personal and work machines which run Linux and macOS.

I upgraded them all to the 23.05 versions from 22.05 by updating my flake.lock and doing a rebuild switch on all machines. No regressions.

Some cobbled together shell scripts and a Git home dir management system (yadm) is what I had before and it doesn’t come close. I also dreaded upgrades with the previous approach. And didn’t have nearly the same control pristine replicated environment across machines.

It’s even flexible enough that on Linux desktop machines I have X11 and bspwm DE environment, on servers no X11, and on macOS it configures my system preferences, Dock icons and installed App Store apps.

Reproducible is key here. I remember trying to manage servers with Ansible, Chef and Puppet in the past but was always disappointed that trying to spin up a new server, or move a service between two usually failed. This is because service S1 forget to declare dependency D, but it happened to work because S2 needed the same dependency and declared it. Or simply that in the past S1 declared D but the config was "cleaned up" and it wasn't realized that D was still needed. (Or it wasn't needed at the time but then needed again, but you didn't declare it).

These types of problems are very rare in NixOS (although I do occasionally see it with the module system, but at least this is config dependent, not system-state dependent). If I apply the same config I have a very high confidence that I will get the same system. I lost my desktop's drive the other day and I was back fully functional within an hour. Most of the time was spent logging into Firefox, Email and various websites rather than installing the system. Managing servers is similarly easy, I would have a hard time moving to something else.

Intrinsic state (like user data or application databases) is still hard, but at least that is the only real concern. Better than dealing with state and config.

> Reproducible is key here.

I like the terms this article used: Divergent, Convergent, Congruent.

https://blog.flyingcircus.io/2016/05/06/thoughts-on-systems-...

In a 'divergent' system (e.g. managed by bash scripts), the resulting system state might diverge from what the bash scripts have, because of manually running commands on the system.

In a 'convergent' system, the system tries to reach a target state by comparing what's there with what it's got.

In a 'congruent' system, the system is forcibly built to equal the target state.

You guys keep saying this word "reproducibility" but I don't think you know what it means. Debian and Arch are more reproducible than NixOS.

In what sense? I don’t think you’ve helped clarify your definition at all

Probably "reproducible" as in "reproducible build" https://reproducible-builds.org/

Yea but nixos has reproducible builds

The term 'reproducible' in 'reproducible build' (https://reproducible-builds.org/) conveys "bit-for-bit identical output". -- The benefit of that is security; others can compile the same sources and check that they get the same binaries.

But more generally, 'reproduce' just means 'create the same thing again'.

Nix uses "reproducible" in a more general sense: you provide the same inputs, and you'll get a program which behaves the same way, regardless of whether you compiled from source or downloaded binaries from a cache.

IMO, I think "reproducible build" is enough to unambiguously refer to the former. But, if you can think of a nicer word for "you get the same behavior from the same inputs" than "reproducible", it would be worth suggesting.

> I can do (almost) all of these things with a traditional Linux distribution, git, and a few scripts written in my language of choice, and I can do it faster.

Is this hypothetical? Or do you have a link you can share?

Many distros have been going through a difficult time just getting reproducible builds. To claim that you can do that (and more) on any traditional Linux distribution so easily is a bold statement.

You can build build bash scripts that roll out your entire system from a full set of source from the ground up in about 5 minutes? I doubt your keyboard could even registers keys fast enough to type all that out.

AND it will be fully reproducible, idempotent, allow rollback, and be readable!

Nix is also cross platform. I have a ZSH and NVim configuration from my NixOS laptop, and when I got a company MacBook, it was extremely simple to use that same configuration and get an identical setup.

That's two dotfiles and two directories each for my setup, which i just rsync over if I feel like. Are you talking about installing necessary depencies or something? I can't think of a ton of examples but maybe a language server? Not that it's strictly part of nvim. Some details perhaps?

Aside: "bated" breath. It's one of those anachronistic words that's only survived by hiding in an idiom.

I was thinking about words like this the other day - words that only occur in regular usage within a phrase. Such words sound odd when they aren’t used in their phrase. Of course I can’t think of any examples right now though

It's an abbreviation of "abated", though, which is still in use.

Many times I’ve horked my system and rebuilding wasn’t quite the next thing I wanted to do. I’ve got a nixos server that I use for my home server and I’ve screwed it up so bad booting was difficult. However it’s super easy to just switch to a prior state. That alone is worth it.

I'm sure you can, but the OP addressed this:

> compared to other distros where discipline is required to maintain reproducibility

You can do these things yourself, but it won't be standard, and it will be prone to error, and it will likely be inferior in many ways to a solution that is maintained and improved by countless people.

A quick Litmus test: can you install Gnome and KDE, uninstall them and not have any leftover package?

None of your listed package managers can do that, and that’s not even a difficult thing like installing 2 firefox versions with only this deep dependency changed.

The thing with traditional package management though, these decisions are not that straightforward, especially for mega packages like KDE or LibreOffice.

When you install a window manager like say, sway, it will install a terminal, alacrity on some distros.

The distro specifically refrains from deleting alacrity when you're uninstalling sway and that's a reasonable thing to do even though people might disagree on it.

> You can clone your system to any machine, and immediately have an identical environment

It's arguably the killer feature of NixOS, if stability and purity means nothing to you.

A few weeks ago I bought a little SBC (Quartz64) for my homemade NAS project. Since I'm already a NixOS user, the bootstrap process was easy:

- Build ARM image for NixOS and boot off the device

- Clone my dotfiles and symlink my config folder into /etc/nixos

- Rebuild my system

And boom. Everything is there, my shell and coreutils and things I've come to expect all get rolled into the system. Updating it just means git pull and a system rebuild. As you say - it's not for the impatient. You have to maintain your config pretty regularly, and covering multiple devices across multiple architectures requires some deliberate config organization.

I'm not sure where I fall on the patience spectrum, but NixOS worked out pretty great for me. It's on my desktop, laptop and homeserver, and I haven't had a single bad update in my 8 months of daily driving it.

How do people deal with isolating system specific config in Nix? Like stuff for a particular graphics card?

I have "if hostname == xyz" in my config, or I use a little config file with flags like "is_laptop".

I use symlinks from configuration.nix and hardware -configuration.nix and point to specific instances for specific systems.

There are a few solutions to this, but I keep separate configs for each device and link them into the entrypoint (/etc/nixos/configuration.nix) as-needed when I set things up. Then I have other "module" files for my terminal environment and my desktop environment, as well as an extra file that enables Wine/gaming packages.

In addition to the other suggestions, you can create different files for (e.g) each machine and use common includes for your apps and settings.

personally, the way I do it is:

/etc/nixos/roles/foo/default.nix has config for the 'foo' role (eg, I have a desktop role that enables all my GUI options)

configuration.nix and hardware-configuration.nix in /etc/nixos/ are symlinked to the actual files in /etc/nixos/hosts/<hostname>/

in the imports section of each machine's configuration.nix, I import /etc/nixos/roles/foo for each role the machine should use. for your graphics card example, I have 'nvidia' and 'nvidia470' roles that pull in their respective nvidia driver (due to an old gaming laptop that requires the legacy driver version)

I have a 'core' role that all machines import, with the global config I want on every host

this allows me to version-control my entire /etc/nixos directory (managed as a private git repo, replicated using syncthing rather than pushed to Github/Gitlab/etc). the symlinks in /etc/nixos are in .gitignore because they're machine-specific, and the actual per-host config files are able to be tracked in their own directories.

I use colmena[1] to manage one nixos configuration for multiple machines: - laptop - desktop - server - rpi nas

I also wipe my entire rootfs every boot with a zfs snapshot rollback[2] using the impermanence module[3] to keep specific stateful data one one of two datasets with regular snapshots: one is backed up with zfs send, the other is just for cache between reboots.

It took a little puzzling to get started, because I didn’t know about the impermanence module at first, so I built my own hacky solution. But I really love this setup. And the way I don’t have cruft to clean.

Also my backups are so much smaller now :’-)

[1]: https://colmena.cli.rs/

[2]: https://grahamc.com/blog/erase-your-darlings/

[3]: https://github.com/nix-community/impermanence

There are lots of ways to do this; pre-flakes e.g. you might have a root configuration file for each host which only includes other files, or have if statements.

Nix flakes lets you define as many configurations as you want, and (if you don't specify one) will select based on the hostname.

I mount '/' as a tmpfs so yes, I nuke my OS every reboot.

It's reconstructed from the /nix/store and obviously my homedir is on a persistent volume.

If this setup could be combined with an easy way to customize things, like ublue (https://ublue.it/) lets you customzie fedora silverblue using Dockerfile configs, I'd adopt it in a heartbeat.

I’d think configuration.nix would be enough?

I have yet to find a non customisable nixos component. It'd a source system that's excellent at building from source so there's always a way.

Sorry, I was unclear, by easy I meant simple. Dockerfiles are simple. I've yet to find an equivalently simple way with Nix.

I haven't tried it but I would suggest looking here for some inspiration https://nixos.wiki/wiki/NixOS_Containers

What does the config for doing that look like?

They are likely talking about using https://github.com/nix-community/impermanence, which is inspired by ideas in https://grahamc.com/blog/erase-your-darlings/

Edit: comments on the latter at https://news.ycombinator.com/item?id=22856199

That's quite fancy, I use use a tmpfs mount for root and a few bind mounts for what I need to persist.

https://news.ycombinator.com/item?id=35390782

https://elis.nu/blog/2020/05/nixos-tmpfs-as-root/

https://elis.nu/blog/2020/06/nixos-tmpfs-as-home/

I posted a snippet before if you feel like checking my history Edit https://news.ycombinator.com/item?id=35390782

thank you!

> Everything in a Nix-based system must be derived strictly from (lockfiled) inputs, making the reproducibility guarantees incredibly strong (barring any network errors or resources being taken down)

Or multithreaded compilation :P

Patience isn't necessarily for time: it's for wasted time.

That's not to say NixOS won't waste your time: it will just waste it in a different way.

I thought "too complex, too much of a timesink" and that kept me from trying it for a long time. My experience is different.

If you are at all idiosyncratic about your setup, using Nix is better than whatever pile of bootstrapping bash scripts you've kludged together over the years.

Absolutely. I'm about a year deep on nix / NixOS and still spend way too much time refactoring things to be more DRY or what I think might be more idiomatic or maintainable going forward (as opposed to a more obvious but uglier approach). Definitely makes it easy to waste time prematurely optimizing, if you're prone to that sort of thing.

The difference is that with Nix, you'd always have a working configuration. With traditional deploy tools, scripts stop working at the most inconvenient timing because they're not reproducible.

alright time to learn nix

And if you don't like it, you might try Guix which is a Guile Scheme based implementation of basically all the same concepts and features.

Wrong. Guix's ecosystem is very much behind Nix's.

This is keeping me from immediately jumping on Nix.

I'd rather use guix and scheme, but it's behind. If I learn nix, and then guix some day catches up, I'll already know nix and won't want to make the change anymore. Can't say I like the outlook for guix.

Interesting. I've been writing Bash scripts for over 20 years and despite my comfort with it, I'm always exploring avenues of replacing it. Despite the time investment already spent, I know there are far better tools out there to do the same jobs.

Perhaps you could expound upon that?

So just like Puppet/Ansible/Salt then?

> Nix is so robust that you could even nuke your filesystems on every log out

Comments like these makes it hard to understand what is really going on.

Persistent storage is kind of the point of using a computer instead of an abacus.

There's been plenty of comments describing the difference between those tools and Nix, and various descriptions and docs are a single search away, so the snide comment comes off as ignorant.

Most of us who use Nix came from one of those tools. I personally went from bash scripts --> Ansible --> Nix.

Nix is lightyears ahead of Ansible. Since I'm familiar with these two, I'll tell you what's wrong with them:

* They are imperative instead of declarative, so you have a ton of extra cognitive load figuring out the order of actions. I have thousands of lines of Nix config, change one file and it's not gonna break because something in another file was installed in the wrong order.

* They are not suited for managing the entire OS, but rather a layer on top of an existing OS. NixOS manages everything from the bottom up, so there are no loose ends to break you.

* They are not truly idempotent. You have to be very disciplined to keep them that way, including Ansible, despite their claims. My ansible deployments broke constantly

* They don't have rollback capability. If you mess up on nix, a single command or keystroke on boot will bring you back to the last working deployment.

* Their DSLs are not powerful enough of an abstraction. Nix is a fully fleshed out language so you don't have to write a bunch of boilerplate like Bash or need some special module like Ansible to make something happen.

* Git support is first class. You can compose your installation from several git repos, so say you want the cutting edge version of a package, you can install the one directly from the maintainer's repo instead of the one in the nixpkgs repo

* Speaking of which, Nix has more packages than any other distro. Key word here, DISTRO. Bash and Ansible aren't distros, and there is no central repo where anything you'd ever want to deploy is available and standardized and denormalized.

And a Nix deployment has persistent and writable storage, not sure where you get the idea that it doesn't.

Ignorant, yes, but in no way snide.

Each and every point listed here could have be taken verbatim from a ten year old blog post why Puppet, or later Ansible, is the new great thing. Being declarative and idempotent is the basic design idea of these tools.

Managing every layer of infrastructure and the entire OS stack is also exactly what they aim for, and ironically also what many people regularly quote as why they haven't taken off more. Git support, and module versioning, is also completely on point. What is it that Nix does differently, and why? Is it a fundamentally different type of tool, such as Puppet is from plain Ruby scripts, or is it a better screwdriver?

That Nix is a distribution in itself is an important difference. But how does that affect the use case?

I realize the question have been asked before, but that's because not many attempts are ever made to explain where it sits in the problem space. It is not written for anyone who has experience with configuration management and infrastructure as code tooling, but instead for passive observers with the intention of bootstrapping the next hype cycle?

I explained specifically using multiple comparisons why it's different from Ansible and Puppet. Several of the guarantees made by Nix were never promised by Puppet and Ansible, such as enforced idempotency and managing the distribution from the ground up. These are fundamental differences, and not a better screwdriver. If you've ever used Puppet and Ansible, you'd know that an OS upgrade will break your scripts. I've had Puppet and Ansible break out from under me countless times. That is basically close to impossible with Nix, because the OS is not under it, it IS the OS. It fully manages anything that it deploys. Puppet and Ansible often just use external tooling like package managers or edit existing config files, which is grossly error prone. Nix IS the package manager, and it IS the config file. This is a huge difference. I understand that you can install a package from scratch and deploy entire config files using Puppet and Ansible, but this is side-stepping the underlying OS's capabilities, and this sort of discipline can only be maintained by a single experienced engineer, and not a large group of people updating the config, unlike Nix, which enforces deployment.

To say that Nix is a better screw driver is like saying that you could write the same level quality and reliability of software using Bash as in Rust. Rust is just a better screw driver than bash - I suppose it's true in some technical legal sense, but most people won't agree, and it's not true in any sense that is useful or valuable.

Again, it's an explanation only a passive bystander.

They're declarative, idempotent, and intended to manage infrastructure, operating systems and clusters, and coordinate application delivery, key rotation and configuration. Repeatability, rollback, and version control are the basic design tenets.

Sorry to hear that an OS upgrade broke your scripts once, but that's hardly the fault of configuration management tools in general or the Puppet/Ansible/Chef/Salt family in particular.

As you might tell from between the lines, I have some experience with these tools in various environments. But I also see the need for another generation, mainly because of the large shift to Kubernetes application delivery. The insistence of these tools to manage the entire OS stack makes people regard them as too complex for what they need. It is my personal belief that the concept of a host is limiting the continued acceptance of this type of tools and I have been spare time experimenting with other primitives to describe state on, such as binary file archives.

Playing with Nix, it is my impression that it is quite similar to other configuration management tools, a host is still the basic entity we serialize state on. State transitions are a linear graph.

What people describe as problematic with configuration management in general is usually perceived complexity, and Nix seems to have the potential for more of that. But I have not used Nix in anger, or in any type of real world complex environment, and I would be very happy to learn from someone who has experience.

> They're declarative, idempotent, and intended to manage infrastructure, operating systems and clusters, and coordinate application delivery, key rotation and configuration. Repeatability, rollback, and version control are the basic design tenets

That's what I'm saying, they are not declarative or idempotent in my experience, at least not Ansible. I'm not as familiar with Puppet so I wont say anything too confidently with that, but Ansible is 1. not idempotent as it allows you with ease to execute operations that can modify the system differently based on the system's state, and 2. not declarative, as the operations are step-wise.

Anyway sounds like you have experience and an interest in the field. My experience with Nix is more personal than professional but I'm happy to try to answer any questions you have about it.

Reply on news.ycombinator.com