Interactive NixOS Tests

I've been running NixOS for almost a year and I agree, Nix is amazing and I love having all my config in a repo, easily reproducible. But god do I miss Arch's documentation so so much.

> And if that's not already bad, I found this section of the docs recently and it said "This section has been moved to pkgs/README.md.", but when I read the README file, it doesn't have what the test package example I was looking for! This is... annoying

That's on me, sorry! Ironically this was the artifact of a pretty large effort to _improve_ documentation [1]! This evidently wasn't executed perfectly, but I was also able to make many desperately needed cleanups in turn.

[1]: https://github.com/NixOS/nixpkgs/pull/245243

> Is this about the official test suite, or general testing with Nix?

Yes, it's talking about the testing framework used by NixOS. See https://nixos.org/manual/nixos/stable/#sec-nixos-tests.

Specifically, the blog post is about debugging the test script like a normal Python program: the standard method to run the test interactively is somewhat clunkier.

Anyway, It's a very niche and technical article about NixOS, I wonder why it was at the top of HN for hours with only 3 bot comments.

> Is this only about Python applications or general?

It has nothing to do with Python, it just happens to use it (previously it was written in Perl). The NixOS test framework is a way to test NixOS configurations: it uses a Qemu VM to run NixOS and a Python script to perform actions (check that a file exists, run a command that should succeed, take a screenshot, etc.).

It's basically an integration test suite for an OS. It's pretty great, if you ask me: you control the entire machine from the boot loader to the desktop environment. You can even network several machines together and test client/server or p2p scenarios.

> How does this compare to testing / debugging flows on Debian, Alpine, CentOS?

I have no idea, but I would be surprised if they have something this sophisticated. Here's some cool ones:

- check that Firefox can open a page and play sounds: https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/fir...

- check that you can install NixOS with full disk encryption and boot into it: https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/ins...

- check that clients can connect to an access point using WPA2 and WPA3 https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/wpa...

- check that a user can download a torrent from a peer behind a NAT https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/bit...

- check that your personal CA certificates are accepted by all browsers https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/cus...

- check that IPsec tunnel protects you from your ISP evesdropping https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/lib...

> When would I care about this?

When you're adding some module code to NixOS, you should write a test with this. More generally, if you're building some complex system with many interacting parts and you want to test it extensively, this is a good way.