A Nix terminology primer by a newcomer
stephank.nl
stephank.nl
I only lasted for a month or two though, for a couple of reasons. The main one was that I had a lot of trouble with Python packages - Nix suggests a certain approach for dealing with them which didn't work very well for me and I didn't manage to set up an alternative that I liked. (For Haskell I loved working with Nix though!) A lot of software I use also wasn't available in the repositories and I tried to build some packages but found the Nix language a bit obtuse. The basic syntax and logic are not too hard but I still found many of the packages I used as examples very confusing.
In the end I'm back to Ubuntu using Docker whenever I need reproducible environments.
I also have the feeling that the tutorials start with the details, instead of starting with an overview so you can understand what is what.
This is guide us already very helpful and I think one should read it before the official tutorial. Thank you!
When you are trying to be as different from everything else as wide-reachingly as Nix is, it seems to work better to build an understanding up rather than down. This seems to frustrate a lot of people because it does not give much payoff for the first few hours of learning (and that's a lot to invest to even understand what something is). The truth is that Nix is really really complicated (expect a year-long learning curve to truly "get" everything), but it's also really hard to take the more-standard approaches to building packages and managing machines seriously once all the pieces finally do all connect. Unfortunately, it kind of makes one feel like being an early adopter of git in a world where no one else uses version control. It's like, "Yes, this is way better, but I'm not going to be able to get anyone else to use it."
This is a fantastic analogy.
I think it's also worth noting that git's UX is notoriously bad for newcomers. These days I'm pretty comfortable with digging myself out of any hole I might mistakenly get into with git. But when I first started using it, all I could do was trivial uses of add/commit/push/pull/merge.
I think the equivalent of "trivial uses of the basics" in Nix would be a "for this use case: just do it this way. You'll understand why later".
I think Guix might be the answer as a more approachable implementation of the Nix ideas, using Scheme for its config language. I haven't spent much time with it but I got going a lot quicker initially compared to Nix.
Out of curiosity, did you tried Guix before trying Nix or vice versa?
When looking at Guix myself, all I see is that the derivations are just wrapped in parentheses otherwise they do look similar. So I'm wondering if scheme actually brings anything here, especially for someone not familiar with a functional language.
IMO Nix language isn't actually that hard, I believe the most complexity comes from nixpkgs and still not adequate documentation of it, so one needs to study its source code.
It’s too bad that guix uses old style mailing lists for development though. I just can’t be bothered setting up myself for that mode of contribution, so I’ll continue to use NixOS for the indefinite future.
I think especially because it's so different you should start with broad concepts and examples, so that the learner can create a broad mental model where they can fill in the details. Otherwise if you hit them with a barrage of details they won't remember any of them later on, because they don't fit anywhere into a broader concept and are disorganized.
Give a few examples that do something useful, show what they're doing differently than other package managers, then break down these examples into the details.
Yes. Reading the official docs, or Nix pills, gives exactly this impression.
I've read some good documentation away from the official docs on how to start using the package manager.
But understanding the overview of how to package things yourself is all wood and no trees.
There's so many moving parts in nix that knowing how they all interact, and which ones are important for your task at hand, is a headache.
function nix-local-build
nix-build --no-out-link \
-E '(import <nixpkgs> {}).callPackage ./'$argv[1]' {}'
end
nix-local-build some.nix
nix-env -i [store-path-from-build]
Question 2 - How to include such a derivation into configuration.nix?
Question 2: including such a derivation is easy with overlays or with the method above.
drv = import ./path-to-derivation.nix;
package = pkgs.callPackage drv {};
After you write a derivation that you can build, you need to create a module [1]. The module defines configuration options, then takes the donation and does other things such as creating service users (if needed), services and configurations.
Then you simply import it and use it.
Regarding question 1 what was suggested by others is what is the current way of doing it, but I will recommend to go through Nix Pills[2]. The author starts with making a simple derivation and then starts introducing conventions. The conventions make things more complex, but they make things more flexible. I can't wait for Flakes[3], they will standardize a lot of this, removing extra cruft and allowing things to be more composable.
[1] https://nixos.org/nixos/manual/index.html#sec-writing-module...
I've even read most Nix Pills posts, but didn't like them much. Too many details about things that I didn't understand at the time of reading, so the why got lost. In Nix you get a lot of details, what is missing is often the why and how. For example I wanted to have a CLI command available with a binary build, finding out how to add the CLI tool to the package through wrapProgram in postInstall wasn't fun. Overall Nix is awesome, but here and there it is hell of a lot frustrating. After reading so many hours I should not have to ask how to install a custom package the right way.
I have overlays in ~/.config/nixpkgs/overlays.nix (though you do this various ways)
My overlay file has the form:
let my-packages-overlay = self: super: { foo = super.callPackage ./path/to/package.nix {} } in [ my-packages-overlay ]
Then you can just nix-env -iA your-package.
This is what always gets me stuck. It seems like lots of tutorials assume you're installing packages by hand still. For me, the whole reason I'm using nix is so that I can use the declarative package management.
I get having local dev dependencies on a per-project basis and running a nix shell to make those dependencies available in your development environment.
But it's just confusing to me how many tutorials suggest running nix-env -i to imperatively install a package. Why would I do that instead of taking advantage of the joys of declarative package management?
So in many cases you are told to use nix-env, because the person just answered how to write a derivation and didn't go further than that.
I think flakes (still work in progress), will help here, because it defines a standard that supposed to do all of that in one file that is easily composable.
- app that is binary , so you can't build it from source, but is using dynamic libraries, then you can use autoPatchelfHook to tell nix to patch the binary to swap out the libraries to nix's version
- app that uses a lot of hard coded file system path that you would prefer not to patch them all out, then you can use buildFHSUserEnv to package/run the application/package in full FHS-compatible scenario
but the main point is that, nix is extreme in its approach, and the best approach in my opinion, is to do swapping step by step, ex:
you want to convert a existing project to nix, lets say nodejs
- you add the main deps in the default.nix (nodejs, yarn, high level utility needed to build)
- use sandBox false to allow building the app with yarn/npm in internet accessible(by default nix uses sandBox which disallow internet inside build step, which causes issue like npm install not working, etc)
- confirm working, then start using tool likes yarn2nix, other similar, and convert the packages itself to a usable set of nix packages
though I do agree that nix have the knowledge bias issue, and alot of my learning of nix involve me looking at the nixpkg sources itself and sometime even the nix's code
Guix?
Even if you build Python-like and JS-like and etc syntax frontends for a hypothetical common backend, they will only be syntactically similar, but the semantics and standard libraries will be different than Python, JS, etc so there's really very little to be gained here.
And maybe it's even worse, since the whole goal is to allow a Python-like frontend to reference functions and variables defined by other frontends, the authors will probably have to understand certain aspects of all frontends. if those frontends are sufficiently similar and familiar (e.g., Python and JS) then there isn't much of a problem, but if you throw in an obscure syntax like Nix this becomes harder.
But 100% agreement that Nix needs a type checker!
The language is imperative, and the result is fully declarative. A pure functional description of this logic would worsen understanding instead of improving it, I believe, because when your conditionals affect multiple output variables, you have to duplicate the condition in multiple places.
Nix advantage here would be that is lazily evaluated, so there would be an additional benefit that only conditionals that are used are evaluated.
Functional benefit of Nix would be that Nix would know that a package doesn't need to be rebuilt if no inputs changed.
Side note Mozilla actually uses Nix internally, it appears that it is used for Rust and Firefox and most likely other things[1][2]
Yes, except that I can read this and I can't read nix. That's probably based on background, but more people are familiar with imperative than functional languages.
> Nix advantage here would be that is lazily evaluated, so there would be an additional benefit that only conditionals that are used are evaluated.
Agreed, although that's unlikely to be unique to nix?
> Functional benefit of Nix would be that Nix would know that a package doesn't need to be rebuilt if no inputs changed.
As would anything that can compare hashes. Or heck, make(1) can do as much.
Of course, language like Python is Turing Complete so you could implement the same functionality . You could also add functions that behave and perform operations lazily.
But you're missing the point though, those are properties that Nix provides natively that are tuned to what is required for this domain. Because the only side effect in nix is a derivation, and because language is lazily evaluated it is made for this purpose.
If you would use for example Python, you would need to write wrappers that would provide that behavior and it no longer would be Python, it would be some kind of DSL written in Python at that point and the risk then would be that it would be very easy to introduce iterative code that's not functional and not lazily evaluated (kind of like it happens with Chef recipes).
The Nix Language actually is not the problem here, the language is fairly simple, the biggest difficulty is actually nixpkgs which is huge and poorly documented. You often have to look at its code to understand what's going on. If it was written in Python you would have the same issues.
There is one thing that Nix Language could do better, a lot of people wish it was a typed language, because if it was typed it would be much easier to find a definitions, and wouldn't require as much of greping the code. But python again wouldn't help here.
Nix may be simple but it’s unfamiliar, and you’re already asking people to understand a novel package management system and everything about everything about their entire dependency tree, and these unknowns are collectively more confounding than the sum of their parts.
Python would help because it is familiar and gradually typed. That said, I’m open to any language that satisfies those properties (maybe also “composes nearly from expressions”).
Regarding the rest of your first paragraph, that's essentially what I said, you can write side effect free code in python, but it's so easy to introduce side effects if you're not paying attention.
I'm saying though that nix language isn't the hard part about nix, but perhaps you're right and it is one extra difficulty that just makes things harder.
But take a look at most o Nix derivations, majority of them end up being just an attribute set (aka dictionary, map etc in other languages)
Take look at https://nixos.org page (they changed it after this submission was posted, and it is better than the old page at showing what Nix can do)
Grafting Python on top of the idiosyncrasies of how Nix does things would be a nightmare of impedance mismatch.
The mixing between parenthesis and braces and the lack of standard indentation makes some derivations really hard to read.
I can't give you an example right now, but I've been looking at the nixos modules that deal with creating NixOS images lately. So you can get an idea of the part of the codebase that gave me this experience.
For sure, you're absolutely right. I'm just saying that doing that in Python would have exacerbated the problem, not made it better. (Imagine someone taking the full power of Python and unleashing it on the derivation writing process, yuck.)
Any official package from Nix belongs to this single repository, making it easy to see what other packages are doing. If I want to see how python packaging is done, I simply search "python" in the nixpkgs repository.
It's also easier to grasp what is going on in nixpkgs than other packaging systems. The Nix expression language offers better readability than say, RPM spec files. Meanwhile the nixpkgs repository includes support for numerous build systems and frameworks. This combined allows you to create packages with minimal effort most of the time.
It's more focused on the result then on how you get there, but that result really impressed me.
yourthing_custom = yourthing.withPackages(ps: [ps.plugin1 ps.plugin2 ] )
Or are you stuck with how to implement it?
Disclaimer: the owner of https://cachix.org
There's also another option, nixpkgs actually has a functionality to generate ec2 image, basd on your configuration.nix[1]
You can also generate other images[2]
[1] https://github.com/NixOS/nixpkgs/blob/master/nixos/maintaine...
[2] https://github.com/nix-community/nixos-generators
edit: I believe you can use [2] even for AWS images, the raw and qcow2 should work on EC2 as well, the [1] is including some additional settings that make NixOS work better there, although there's nothing stopping you from adding these to your own configuration.nix
Also, this is not really a fault of Nix, but in EC2 if you import image it has to go through their conversion mechansim, which takes about 10 minutes :( A kind of workaround around it is to have another instance with attached extra EBS drive, use dd to write raw image on that disk. Then disconnect the EBS, create a snapshot of it and convert the snapshot to an AMI.
But I just tried, and it did work for me. The generated image was provided as the last line of the output.
It looked like this:
Making image hybrid...
/nix/store/ra02s6nmhfklmsrmhkd960xdqpwghgak-nixos.iso/iso/nixos.iso
BTW: I believe NixOS image generation can be run only on NixOS machine, because it needs to run a VM and that requires some kernel modules to be loaded. At least I never was able to made it work on OS X.
The nixos-infect code though will turn your Ubuntu installation into NixOS, so you might try to boot to your Ubuntu machine, convert it to NixOS and I'm hoping then nixos-generate might start working.
Note though that "do" didn't work either for me, and I didn't look to investigate why, I figured out that maybe it was because I was running it on machine on VirtualBox and not DO.
We build and distribute Ardour (a cross-platform open source DAW) in a single package that runs on every Linux distribution that has libc, libstdc++, and some version of Xlib.
Every Linux distribution that is, except NixOS. They decided to patch the runtime linker so that the approach usable on every other Linux distro breaks there.
This says something about NixOS, and presumably Nix too, though I'm not sure what.
For anyone curious, this wiki page describes how to package and run some precompiled binaries on NixOS, particularly the section "The Dynamic Loader": https://nixos.wiki/wiki/Packaging/Binaries
NixOS tries to insist that programs reference only other paths within the nix store (e.g. /nix/store/9rabxvqbv0vgjmydiv59wkz768b5fmbc-glibc-2.30/lib64/ld-linux-x86-64.so.2 which is a specific version built by a specific compiler, rather than /lib64/ld-linux-x86-64.so.2). This is the source of the per-program isolation that enables all the nifty features of NixOS.
NixOS _does_ make exceptions to that though, for example providing both /bin/sh and /usr/bin/env at those paths.
Note that the way we do things is not unique - Firefox has been packaged this way by the Mozilla Foundation for years. We consider ourselves closer to a "traditional ISV" than merely the provider of source code to the open source community (though obviously, we play the latter roll too).
We don't have the resources to spend on a nixOS-specific package, nor the desire. Fortunately, of course, nixOS users can create suitable packages for nixOS, and I think they are.
It just says something that out of the myriad flavors of Linux, nixOS is the only one that has broken this rather old concept for packaging software from ISVs.
But please, carry on! Seems interesting enough.
They do eliminate any ability to automatically find libraries, and the dynamic loader does not exist at the path where it exists on most other distros, which means binaries linked elsewhere will not run unmodified on nixos.
The binary's .interp section indicates which dynamic linker must be used. On a regular Linux distribution, you will find that this is a global path:
$ readelf -p .interp $(which cp)
String dump of section '.interp':
[ 0] /lib64/ld-linux-x86-64.so.2
Instead, the dynamic loader of the glibc version that a program was built against is used:
$ readelf -p .interp $(which cp)
String dump of section '.interp':
[ 0] /nix/store/jx19wa4xlh9n4324xdl9rjnykd19mmq3-glibc-2.30/lib/ld-linux-x86-64.so.2Edit: I assume you can't do it very well system-wide because individual packages might need conflicting versions. Okay. Has something like runtime injection been considered? What are the trade-offs here? (I think it's fairly obvious that I'm not very familiar with the whole matter yet).
For example, the built-in firewall is fairly basic (at least for running on a router; it does nearly everything you'd need for most desktop and server applications). I run NixOS on my router, so I needed something more. So, I wrote https://github.com/thequux/nix-zone-firewall (note: the readme is slightly out of date, but it does give you the gist of it). First, I built a way to declaratively put rules in different chains (core.nix), and then I built a zone-based firewall on top of that. I then have another layer of configuration options in my router's config (not that repo) to be able to spread configuration across multiple files.
Even though this was the first significant bit of Nix that I wrote, I still was able to put the entire thing together in a single evening after work.
I would also love to see a tutorial for beginners on Linux that covers common tasks like viewing changelogs, creating a simple package, and pinning multiple dependencies in an environment.
I didn't find the direct comparison to apt to be useful before installing nixos, to be honest, but as primarily a `nix-shell --pure` user who just got nixos installed on my server, I suppose I might revisit it...