How to Learn Nix
ianthehenry.com
ianthehenry.com
But once it starts working: how does anyone do this any other way?
- The language is reasonably well-documented and those docs are rarely wrong.
- Learn bash, like really really learn bash.
- Check out nixpkgs and use that to figure out how to do stuff.
- Only do this if paying the entire cost in time and frustration in one aneurism-inducing spasm upfront is a worthwhile tradeoff for having everything work perfectly every time thereafter.
It’s kinda like being Paul Atriedes: you get magic powers but first you put your hand in the fucking box. What’s in the box?
“Nix!”
This is on top of a GUI (syncs with git repos, at minimum about:config like with the documentation inline and ability to edit free-form entries), a Nix-style installer (pulls your config from a git host, inline machine profile editor, formatting help and saves formatting to repo), maybe some languages improvements, etc
This is my current list. I add them as a submodule to a repo.
ag -A10 'inputs' -G flake.nix
quidome/nixos-config is no longer available though.
but as others said, a repo aggregator like dockerhub for nix configs'd be great awesome (not saying you should make it or anyihign)
Hopefully that'll come in the next ~10 hours, or maybe I need to switch to one of these other learning resources
- Tutorial motivated by accomplishing things [0]
- Learn nix language syntax [1]
- If video is more your thing [2]
- The official "bite-sized guide" Nix Pills [3]
[0] https://nix.dev/
[2] https://www.youtube.com/playlist?list=PLRGI9KQ3_HP_OFRG6R-p4...
https://ianthehenry.com/posts/how-to-learn-nix/builders/
The Nix manual just doesn't talk about this. But the Nixpkgs manual will, eventually. Here's what it has to say:
https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-mkShell
That's it. Nix Pills does cover making development environments in chapter 10:
https://nixos.org/guides/nix-pills/developing-with-nix-shell...
But Nix Pills is not really a "bite-sized guide" so much as a "let's motivate and invent the Nixpkgs genericBuilder infrastructure from scratch." So it doesn't really show you an example of how to write shell.nix files.
Writing reproducible dev environments per project ("shell.nix files") is the main reason I use Nix. But Nix does not... make it very easy, out of the box. There are two more posts (separate from the documentation-reading sections) that will explain how to make this more ergonomic:
- https://ianthehenry.com/posts/how-to-learn-nix/saving-your-s...
- https://ianthehenry.com/posts/how-to-learn-nix/nix-zshell/
(This is slightly unfair because if you're using Nix to build software, not just create a development environment, you get nix-shell for free. mkShell is sort of a special case of "I don't want the final result in the /nix/store; I just want you to put me in a shell with these packages installed." It's not a case that Nix optimizes very well for, even though it is my primary use case.)
Finally, obligatory caveat that everything is completely different now in the "new" (unstable, flakes-based) Nix interface:
> Since "How to Learn Nix" mirrors the (bonkers) presentation order from the official documentation
Yes, I should have clarified that the complaint is directed at nix docs and not your blog.
> I am afraid to say that you will not learn how to do this until part 32, deep into the Nixpkgs manual.
I was actually feeling a little dejected after my first sprint at learning this. I had skipped ahead to your summary posts ("so I read the manual") and saw that I wasn't even close to learning how to use nix as I wanted.
Some of the comments here actually gave me hope that the juice will be worth the squeeze, so I'll jump back in tonight. I might switch to `nix.dev` tutorial that will hopefully get me moving a bit, and then jump back to your blog and the official docs.
[0] Really, having your stream of conscious helps a lot. Sometimes you explain something a little more in depth or from a different angle, other times your confusion reassures me that I'm not missing something.
I saw you say elsewhere that you feel like these are "low effort" posts compared to your other writing, but it also feels like it's filling a whole in the online-learning landscape. So again, thanks!
I wrote How to Learn Nix mostly out of curiosity: I was interested in Nix as a concept, and I didn't really know what it could do going into it. I picked the title long before it became clear that reading the official documentation was not a good way to actually learn how to use Nix. "The Nix Diaries" or something would be a lot more accurate.
That said I don't know what a better approach would look like. nix.dev and tutorials like that are very useful, but my (possibly outdated) understanding is that they never take a step back and explain how or why things work. So you can follow along with what they're doing, following a carefully lain road, but as soon as the road ends you find yourself in the middle of nowhere without a map of your surroundings. (This was how I initially tried to "learn Nix" in 2016, and basically gave up on it -- nix.dev didn't exist back then, but I followed various random online tutorials.)
I think it's a good idea to come at it from both sides, and How to Learn Nix is much more in the "build intuition" camp than the "get something done" camp. I'm glad you're finding it useful, though!
> Some of the comments here actually gave me hope that the juice will be worth the squeeze, so I'll jump back in tonight. I might switch to `nix.dev` tutorial that will hopefully get me moving a bit, and then jump back to your blog and the official docs.
FWIW, my learning journey with Nix was copy-pasting snippets and casually getting a deeper understanding as I needed it over months.
The experience with Nix is 95% wonderful, 5% painful. With many other tools, you can use them without really needing to know what's going on. With Nix.. you'll likely end up wanting to do something which will demand you understand stuff. -- Time spent reading the manual isn't wasted.
It's ridiculously hard to get started. The online resources are simply inadequate: incorrect and/or out-of-date information is basically equally common as useful information. There isn't a good book you can get on Amazon. The "Learn" boards thing is full of "you're doing it wrong" bullshit. The Discord is so-so at best for getting help.
For me personally it was somewhere between 1-3 months of being frustrated basically every day and I still want to throw things sometimes. It's basically clone `nixpkgs`, figure out how to get the REPL going, painstakingly reading the code of the stuff that does work, and pain.
It says absolutely nothing about how smart/capable/motivated/senior someone is that a week or two in it's like "fuck this".
The payoff is outrageous, but the barrier to entry is nowhere near something that can be called "acceptable", let alone "batteries included". I completely understand why a lot of people are all-but-hostile to it.
I would budget a minimum of a highly frustrating month working with it every day to start to having the lightbulb moments, and frankly your use case needs to be pretty extreme in one way or another for that to be worth it. A lot of people don't have a friggin month to screw around with a fairly niche infrastructure if what they're doing now works for them.
I really, really hope that at some point someone gets serious about making this fairly magical thing even remotely learnable, but right now it's a fucking mess.
We need to come to terms with the fact that the way we as programmers acquire information about our tools is in a dire need of help. Search indices are just not valid references for us. We should never have learned this method of learning in the first place, but it was easy and painless in the beginning.
People need to just slow down a bit and become more methodical one thinks.
I think if there were a vibrant StackOverflow community around Nix it would go a very long way to making it more usable.
That's because all the ways you learned how to do things on linux you need to throw out of window and start from scratch.
For example, I'm using it to describe a set of containers and services in my infrastructure, and it allows me to properly describe the interfaces between the different components of the infrastructure, using several modules : - a "base" module defines the options used for describing the services - one defines the services themselves - one handles the reverse proxy configuration for these services (for multiple frontends), as well as the SSL certificates and DNS zones - one handles the containers themselves (through Docker or LXD) - one handles the networking overlay - one handles the "shared services" (databases, caches, etc)
Each of these modules is basically a function that takes the resulting configuration as argument, allowing it to reuse values defined by the other modules, and the configuration itself is the fixpoint of the composition of all of these functions.
It all composes quite well, and the nixpkgs module system allows me to document the options provided by each module. I can even reuse some of the existing NixOS to create a web interface where other admins can search for configuration options and lookup their documentation. Some things are exported in JSON, some are used directly as NixOS modules, some things source the inventory for Ansible playbooks, etc.
It allowed me to create a well-defined source of truth for a very heteregenous architecture, while preserving some of the existing workflows and tooling.
It's not ideal of course : Nix itself is not typed, the nixpkgs module system provides some typing, but the integration is not perfect, and this makes error handling quite tedious. Also, the interpreter is quite slow, there is no documented way to embed it into your own program, and I
- Examples that just don't work.
- Commands that run and no output is given, even though the text I'm following says it should output information I need.
- Do I need nix-darwin? Well, the readme explains what it is: "Nix modules for darwin, /etc/nixos/configuration.nix for macOS." What? (Finally I found an answer at Stack overflow, and it seems I don't need it. Seems to be kind of NixOS on top of macOS)
- Can't build with nix-build. After long time searching it seems like you're supposed to run nix-build -E 'with import <nixpkgs> {}; callPackage ./default.nix {}' - shouldn't there be a simpler way for this common case?
- An issue open since 2018 that the manuals simple example doesn't build: https://github.com/NixOS/nix/issues/2259 People chiming in years later that they only could build the example after reading the suggestions in that issue.
- And now I can't start my nix-shell because it says there's a problem with SSL certificates. I'm giving up.
This is because the default.nix you have is exporting a lambda that expects arguments to be saturated, `callPackage` does this for you automagically hence why you have to use it.
Instead you probably want the nixpkgs import line _inside_ default.nix, either in a let-in binding or using the `?` operator to default a lambda argument to something, usually nixpkgs itself bound to a `pkgs` argument so you can use `pkgs`[0].
Even better, you can use Niv to pin nixpkgs to a specific nixpkgs commit so that it doesn't change as you update your system's nixpkgs channel with the `nix-channel` command because `<nixpkgs>` is special syntax referring to what's stored in your $NIX_PATH[1].
[0]: see the default argument section: https://nixos.wiki/wiki/Nix_Expression_Language [1]: https://nixos.org/guides/nix-pills/nix-search-paths.html
I'm about to dive headfirst into this nix business and this is intimidating, lol
Fully agree, but the given example isn't such a case but rather incorrect code.
The example is somewhat akin to writing the following Python code and expecting it to print "hello".
def main():
print 'hello'
I understood that an argument was needed, that error was clear and when I googled it I found the solution that I wrote.
I supposed it's meant to plug in to something bigger maybe and that's why there are arguments without import everywhere.. maybe? But I've given up now, so I'll never know, we're just gonna suck it up and keep using brew and try to make it idiot-proof somehow.
It just feels like a shame that nix is so unapproachable. It's just as much apple's fault though - if xcode has some open command line tools I'm sure there would have been ready-made formulas for everything we needed. At least I got to vent.
It's on the NixOS website, but IMO it's worth making it more visible to newcomers:
https://nixos.org/guides/nix-pills/
From my experience, it became considerably more easier to figure out or search for solutions once I've grasped the basics from "Nix Pills." Once I became comfortable enough, I also relied heavily on the Nixpkgs repository[1] as a great source of information. It has an approachable codebase and some quick grepping gave me many examples when trying to figure out how to write packages for a particular language.
> I understood that an argument was needed, that error was clear and when I googled it I found the solution that I wrote.
So in this case the advise was worst than useless? Had you been figuring things out on your own, would it have been less bad?
> It's just as much apple's fault though - if xcode has some open command line tools I'm sure there would have been ready-made formulas for everything we needed.
Yes that is too true. Keeping things working with apple is so draining it's hard to have energy left over to do other things right.
I think the StackOverflow post linked in the answer gives a good explanation: https://stackoverflow.com/questions/44088192/when-and-how-sh...
With other languages, e.g. with Haskell, you can run "stack build" and it will build the project.. or with C, you can run "cc *.c" and it will build an executable with main. -- Whereas, the `default.nix` example given in the tutorial is supposed to be given to callPackage. (The tutorial which provides the example instructs you to modify the all-packages.nix file in a clone of the nixpkgs repository).
...Which is unhelpful with the mindset of "I just want to write a Nix package for my thing", or just weird compared to what you'd expect from "Hello World". (But makes more sense in terms of "how to get started with nixpkgs").
The UX is much better with Nix Flakes. -- With flakes, there's a standard file format where you can run "nix build" and expect it to work.
The flakes are essentially supposed to standardize it: https://www.tweag.io/blog/2020-05-25-flakes/ (well their goal is to fix other issues, but the standard format of the file fixes that problem too and CLI supports it as well)
default.nix (i'm guessing):
derivation {
name = "simple";
builder = "${bash}/bin/bash";
args = [ ./simple_builder.sh ];
inherit gcc coreutils;
src = ./simple.c;
system = builtins.currentSystem;
}
nix-build -E 'with import <nixpkgs> {}; callPackage ./default.nix {}'
default.nix:
with (import <nixpkgs> {});
derivation {
name = "simple";
builder = "${bash}/bin/bash";
args = [ ./simple_builder.sh ];
inherit gcc coreutils;
src = ./simple.c;
system = builtins.currentSystem;
}
nix build
0: https://nixos.org/guides/nix-pills/working-derivation.html#i...
yes, the nixpkgs changes and some old articles might no longer be accurate :/
> - Commands that run and no output is given, even though the text I'm following says it should output information I need.
could you give an example?
> - Do I need nix-darwin? Well, the readme explains what it is: "Nix modules for darwin, /etc/nixos/configuration.nix for macOS." What? (Finally I found an answer at Stack overflow, and it seems I don't need it. Seems to be kind of NixOS on top of macOS)
yes, you don't need it for what you described. nix-darwin is basically to provide configuration functionality of NixOS to your Mac. Nix-darwin is 3rd party product that's written in Nix for Nix. Similarly Home Manager.
> - Can't build with nix-build. After long time searching it seems like you're supposed to run nix-build -E 'with import <nixpkgs> {}; callPackage ./default.nix {}' - shouldn't there be a simpler way for this common case?
a lot of people wrote about it. This is a convention that evolved from nixpkgs, it's still not officially supported by CLI tool (that's why the command looks so complicated, it's similar to calling `python -c "<whatever>`), seems like the default.nix file which is very similar in concept to setup.py (as you can put whatever you want inside) is being replaced by more stricter flakes.nix file. Which in a way brings some similarities to the callPackage convention.
> - An issue open since 2018 that the manuals simple example doesn't build: https://github.com/NixOS/nix/issues/2259 People chiming in years later that they only could build the example after reading the suggestions in that issue.
Because software evolves and old versions of file like to disappear with time. I run into this as well, although it was nix pills documentation. My solution was to simply get the new version of hello and use that. Although I agree, that's still unnecessarily confusing. I think a proper solution would be to host the file on the own server. Or even better, make a basic hello.c file and list its content in the doc.
> - And now I can't start my nix-shell because it says there's a problem with SSL certificates. I'm giving up.
You most likely uninstalled nss-cacert package. Perhaps `nix-env --rollback` would help.
As someone who tried Nix-- never understood what the big selling point is-- what can Nix do my existing tools can't? (without me spending 6 months)
Nix can create isolated environments, but as far as I see, you can only install the tools Nix repo supports. And the official examples only give toy examples --here's a simple python script you can run with Nix-- but I can just run this simple script in a virtual env, or docker if I have 2-3 different scripts.
What problem is Nix solving? What pain does it fix that is so important it would justify me spending 6 months on it?
And then there is NixOS, which again, I have no idea-- I can create my own Linux distribution that is what, reproducible? Again, is this a real problem anyone faces (and that isnt fixed by docker or ansible or any of the other hundreds of tools)?
My point is: Why is Nix getting so much hype when I dont see them solving any real problems? Sure, they promise a lot, but I have yet to see any real solutions.
Hence the Nix Hype Machine
> Reproducible builds and deployments
> Learn how to make reproducible, declarative and reliable systems.
I can then use that same environment to do CI without even creating a docker image, e.g. this for Gitlab CI:
image: nixos/nix:latest
before_script:
- nix-env -f shell.nix -i -A buildInputs
You can also use Nix to control what is on a machine. E.g. for our lab setup we had a gateway machine behind which was all the test equipment. NixOS was great for having the exact config of that machine checked into version control so we could recreate it. It's far more reliable than Ansible or something like that.
- everything from kernel and drivers up is specified declaratively in a uniform, expressive way. You version control your entire infrastructure. Not in the sense of a Dockerfile. The Ansible stuff, the Dockerfile, the build of your library, the build of your own software is in one language that can happily and easily reference each other as a unified system.
- with a few caveats, you are cryptographically guaranteed to get the same result, every time, no matter what. if it worked once, it will never break again.
- it has mechanisms reminiscent of git or something: you can checkout a different computer or set of computers, and if you don’t like the result, revert to what you had before.
- it treats patching of arbitrary software as a completely typical and first-class activity. if anything doesn’t work how you want, you write a patch, check it in, and it works now and always will. upstream at your leisure or not at all.
- you get exactly what you want on your system, no more, no less. no gnome-keyring-daemon bullshit on your headless server Ubuntu 20.04 LTS.
- caching of artifacts both locally and via cache servers run by you or others or both is very granular. no giant slabs of docker composition towering ever higher.
- I have machine configurations that if say a HDD dies means I can just replace it and be pretty much where I was an hour later from scratch.
- I can update a configuration (even doing the equivalent of a major OS update) without fear as it's possible to just use the old configuration if something is broken.
For Nix in general:
- Can define versioned development environments which aren't awkwardly sandboxed as they are in Docker.
- It's possible to make multi language/ecosystem builds.
- Builds only build what needs building.
- You can build on one machine and ship the build to another as if that machine had built it.
Right.
The benefits are clearer for DevOps than for developer workstations. (e.g. Nix makes it easy to declare the set of packages installed, with atomic rollbacks).
For developers, I think it's still worth keeping an eye on, since Nix provides a programmatic way of dealing with packages of software; and dealing with packages of software is pretty much what a developer will need to do when driving their system.
e.g. project READMEs typically list some commands for setting up a developer environment; Nix promises a consistent, single command for that (if projects provide Nix files).
For most things that you can do with Nix, there's a quicker + dirtier approach already. e.g. VSCode allows isolated/ephemeral dev environments with remote containers; Docker provides an easy way to be confident software will run a certain way in other system; ZFS/btrfs snapshots allow for rolling back changes; nvm/rvm/asdf allow for project-specific versions of packages; apt-get/brew provide a way of installing packages to a system...
Nix is an elegant and coherent solution to these kinds of problems.
> Nix can create isolated environments, but as far as I see, you can only install the tools Nix repo supports.
Not at all! -- It's comparable to e.g. Docker: With Docker, you can run containers using images served from DockerHub.. but you also have the option of building Docker images and using those.
With Nix, you describe the packages/libraries/tools you want with Nix. This varies from trivially easy, to "have to really understand what you're doing" difficult.
If you do very typical things that are within the sweet spot of the quick and dirty solutions, you'll likely see nix as redundant.
If for some reason, the quick and dirty solutions always seem to break for you you'll grow to love nix and likely NixOS.
Well, for one, someone send me a random Python script with a dependency on a PostgreSQL driver (that probably needs to compile some C library). Just the script, no `requirements.txt` or `Pipfile.lock` or `pyproject.toml` files declaring the dependencies. I could have just created a virtualenv and tried to install it using pip, that would probably fail because I don't have a proper C compiler setup. Or maybe I have but it is missing headers. Maybe it will work because there is already a wheel for my system, but maybe it will not because some other random reason.
Instead, I just wrote two lines for shebang:
#! /usr/bin/env nix-shell
#! nix-shell -i python3 -p python3 python3Packages.psycopg2
> What problem is Nix solving? What pain does it fix that is so important it would justify me spending 6 months on it?
Really the meme that you need to spend months just to use Nix properly needs to die. I literally skimped the docs and got a working NixOS install at my first try on the system [1]. The default configuration is really well documented with examples, and I think most people can get what they want quickly.
Eventually I wanted to use something that was not packaged on nixpkgs, or I started to want to upgrade some specific packages without using unstable. At that point it was when I started to think Nix was kinda of painful (not because it was difficult to do, however the documentation is not good and you end up searching a lot until you find some answers). At that point yes, it is kinda a journey, but it is very similar to learning a new language: you eventually can get very proficient at it, but you never stop learning.
> And then there is NixOS, which again, I have no idea-- I can create my own Linux distribution that is what, reproducible? Again, is this a real problem anyone faces (and that isnt fixed by docker or ansible or any of the other hundreds of tools)?
Try to have reproducible system with Ansible. Yeah, I tried. It simply doesn't work well. Ansible is always a pain because steps randomly fail in a new machine, and you can waste hours of debugging for simply issues. Not NixOS: if your configuration builds there is a great chance that everything will work.
NixOS is the only system that I am sure that I have the exactly same configuration as my other systems, as long I am running the same commit from my nix-configs repository. Can't say the same for Ansible: if I setup a machine using Ansible right now and 2 days from now, pretty sure they will have different package versions (since the repository will have changed them), for example.
Also, you don't have rollbacks with Ansible. Yeah, you can have ZFS/Btrfs taking periodic snapshots and use this for rollbacks. This is not the same though. The UX in NixOS is simply so much better (reboot your system and select which generation you want to boot).
> My point is: Why is Nix getting so much hype when I dont see them solving any real problems? Sure, they promise a lot, but I have yet to see any real solutions.
Well, my point is: Nix solves real problems. Maybe you are not bitten as much as I am with them, maybe you think that spend half a hour trying to get a random Python script to work is fine. And I completely concur with you. It doesn't mean that Nix is not worth your time though.
[1]: Now, I think the fact that I used Arch Linux before really helped, since both NixOS and Arch Linux have an installation process that is completely terminal based instead of a proper installer.
NixOS and Guix present alternative systems that reuse functional developer skills. If you don't have or want those skills then they aren't going to make any more sense than choosing Smalltalk or a lisp machine while being resistant to leveraging their respective paradigms.
Nothing; even CSS is Turing-complete. It's all a question of how easy, how secure, how reproducible, etc.
> as far as I see, you can only install the tools Nix repo supports
Lol what? Nix literally just executes a given command, with given arguments and given environment variables. This comment is like saying "Makefiles can only build C projects".
Just like Make, if you can build something with some shell commands, then you can build it with Nix. It's often as simple as:
runCommand "my-program" { myEnvVar = myValue; } ''
put your shell commands here
mv my-build-product "$out"
''
> Again, is this a real problem anyone faces (and that isnt fixed by docker or ansible or any of the other hundreds of tools)?
Nix has been around for two decades. It's more appropriate to ask what was the point of creating Docker, Ansible, virtualenv, etc. when Nix already existed.
> And then there is NixOS, which again, I have no idea-- I can create my own Linux distribution that is what, reproducible?
Maybe you're getting hung up on terminology; if you would consider "Ubuntu, but with my SSH config file" as "my own Linux distribution", then sure, NixOS creates Linux distributions. If you would call that "configuring a Linux system", then NixOS is for configuring Linux systems.
Nix, after two decades, still has an incredibly sharp learning curve, especially to build your own packages. Docker has almost no learning curve, including creating your own images and sharing them with the world.
with import <nixpkgs> {};
runCommand "my-thing"
{
someSortOfSource = fetchurl {
url = "https://example.com/thing.tar.gz";
sha256 = "0000000000000000000000000000000000000000000000000000000000000000";
};
buildInputs = [ gcc python3 whatever-else-you-like ];
}
''
tar xzf "$someSortOfSource"
cd whateverWasInThatTarball
run whatever commands you like
mv whatever-the-result-should-be "$out"
''
> Docker has almost no learning curve, including creating your own images and sharing them with the world.
It took me several attempts to learn Docker, and I'm still very much a newbie with it. To be fair, most of the difficulty I had was due to my misunderstandings; e.g. "surely nobody actually does that?" (e.g. "download entire OS just to run a command"; "ignore all the hashing and just fetch 'latest'"; "run 'apt-get -y update' without having any idea what it's going to do a week from now"; etc.)
Once Docker 'clicked' for me (i.e. I realised that indeed, people are doing that sort of stuff), it became much easier. It's also when I completely lost interest in Docker, and started using OCI containers instead (you can get remarkably far with just 'tar', 'sha256sum' and 'jq'!).
These days my only interaction with Docker is when other people's containers cause an outage at work, due to Docker blowing through all the disk space on our cloud servers. (Fun fact: you can resize AWS disk partitions on-the-fly, but not if they're 100% full, since the partition-editing commands try to allocate temp files)
I'd say that the relative popularity of the tools is an indication of how well suited they were for the things that people actually wanted to do.
Nobody (exaggerating here for dramatic effect, but it probably applies to a large part of the unseen 99%: https://www.hanselman.com/blog/dark-matter-developers-the-un...) wants to have fully reproducible builds and a unified language for configuring everything like Nix attempts to do, or even have an OS that attempts to centralize those concepts, like NixOS does because a lot of it is alien to them.
They just want to run their PHP CRUD app in a vaguely predictable way that is less painful than what they've historically done (ship scripts through FTP, experience pain due to php.ini being different, or a system package not being present, or the system package being of a different version). Docker and other solutions like it are the closest that they can get to shipping their machine to prod, given that shipping VMs has more challenges, than just telling their clients or even their ops to just run a Docker container with some configuration.
Nor do they want rootless containers, or distroless containers or Podman or any of the other fancy solutions - they just want their Ubuntu userland and their app to sort of work, most of the time.
We had containers in one way or another long before Docker, for example, FreeBSD jails were pretty good. And yet, they never caught on, nor did FreeBSD outside of large corporations, because for the most part the developer experience just wasn't there. Then came along Docker and changed everything (with plenty of bumps along the way), now building and running your app is pretty simple:
1. nano Dockerfile
2. docker build -t some_image_tag .
3. docker run some_image_tag
(or, you know, push the image somewhere)
Additionally, most developers don't get paid for ensuring that their application will run in a stable and predictable manner 5 years from now. Most developers get paid for shipping that mediocre piece of software that their boss wants them to before the end of the week and having it work well enough so the company can earn some money, before they're told to move on and do something else the next week. I'm not saying that it's how things should be, but just the way they are, at least for a lot of people who don't tackle interesting problems yet still put food on the table.
That is the "tragedy of good enough" and why Nix, NixOS, jails, FreeBSD or anything else won't be as popular until vaguely low skilled developers can get easily started with them. Maybe things just need to cool down and become more stable, even Kubernetes has gotten less horrible thanks to Rancher/K3s in the last years, maybe in a decade package management and the way OSes are will be more approachable.
Right. The Dockerfile format is much less elegant than Nix, but it's easier to get started with, and can be used for most cases.
I don't necessarily see everyone _writing_ Nix. But I think one way Nix can gain in adoption with developers is Nix makes it easy to make dependencies available for working on a project.
I'd also point to git: many developers struggle to use git; and might use it for some time without an intuitive understanding. The network effects of GitHub have somehow been enough to make git a prominent tool, despite it being difficult to use.
I think as more Nix stuff becomes copy-pasteable, there are enough use cases which don't require deep understanding of Nix.
Nixos extends this reproducibility up to the level of the OS. Rather than a machine being a giant ball of mutable state, some of it pushed around by configuration management, Nix pulls the entire system state up to a single top value which is set by the bootloader (or similar high-level mutable cell). Like switching to functional programming, at first it just appears different, difficult, and not compelling. But after you start to understand the paradigm, a lot of nice capabilities just fall out, that would otherwise be effectively out of reach.
[0] Not deterministic builds per se, which is an ongoing effort.
I find it disgusting you talk about an opensource effort of mostly volunteers like this. And I think you are wrong. There is no machine: it's people.
Yes it's hard. It might not be for you. But it is cool to some people, so they talk about it.
Other replies to your comment answer your questions well (showing people really care).
This is not bought marketing BS but some "agency machine". This is the labor of love of many. And you are simply dismissive.
But overall: it makes me more productive. It is my secret weapon to manage the complexity of software development. The benefit is not on day 1, but on day 100 and day 1000.
0. Give up the expectation of figuring it out quick
1. Read the official manual instead of tutorials
2. Read example configurations from public repos
That said, some tech goes too far. You'll never learn C++ by reading the standards.
Also, IRC.
> By default, `args` is a set of derivation names denoting derivations in the active Nix expression. These are realised, and the resulting output paths are installed.
?? - I think the author really hits the nail on the head with this point.
For example, determining if and how your desktop environment can enable Wayland support.
Written by the same author if you didn't notice. In fact, reading over ianthehenry's blog is an interesting adventure in one mans experience learning this new tool.
but there doesn't seem to be a single agreed upon replacement for it. I wrote my own, I'm sure others have well. It seems home-manager is popular, but it's certainly not even close to universally adopted and I've never gotten around to learning it myself.
Maybe nix-env -irf does the Right Thing; certainly nobody pointed me at it when I was battling against the evil that is nix-env...
aidenn0, could you explain what `-irf` does and what it should do? I'm curious now, even though I don't currently use Nix.
The problem is that it lulls you into thinking it serves the purpose well enough and then eventually things break. Then they get more broken. Then you can't figure out which thing you installed is breaking things. Then you finally do and can't figure out how to uninstall the thing that broke things.
> aidenn0, could you explain what `-irf` does and what it should do? I'm curious now, even though I don't currently use Nix.
I'm pretty sure it sets up the environment from a file; I found it one link away from the link in the comment I replied to here[1]
1: https://ianthehenry.com/posts/how-to-learn-nix/declarative-u...
I'd rather phrase this as: the 'problem' with `nix-env -i` is "I forgot I did that" or "I didn't know I did that".
I believe since the command is the easiest thing to do with Nix, and many people will try it and maybe got back to non-Nix package management, it disproportionately leads to newbies asking for help; thus, the general mood against it.
1. Imperative package management is evil
2. The package name-based interface used by nix-env is evil
I don't ascribe to (1); I quite like installing things "the old-fashioned way." Saying "install this thing" is a very nice, familiar UX. (Even if under the hood you implement that as "add this to my declared list of packages and then rebuild the world," I would still rather run a single command than edit a text file and then run a command most of the time.)
But I think that (2) is true, for the reasons outlined in that post. I have never really heard anyone defending name-based operations; the universal consensus is that it was a bad decision made decades ago. It's something that you can avoid if you are aware of the problem, but it is still, for backwards compatibility, the "default" way that nix-env works. (And although you can install packages the "right" way, you have to resort to name-based operations if you use nix-env's built-in uninstall or upgrade commands, which is also very confusing.)
nix-env -irf is neither (1) imperative nor (2) name-based; I use it as a way to implement my home-rolled declarative package management solution linked in the sibling, taking advantage of the fact that nix-env is okay at building profiles already. The flags literally mean "uninstall everything, then install everything that is listed in this file," but what that translates to is "create a new profile with the contents of this file, disregarding the contents of the current profile."
If you prefer imperative programming, then think of it being used like this:
const result = defaultNixpkgsSet;
for (overlay in overlays) {
// Merge the overlay's return value into the results value
result := result.merge(overlay(
result, // Give each overlay a reference to result
result.copy() // Also give a snapshot of the contents so-far
))
}
return result;
function(result, snapshot) {
return {
maven = snapshot.maven.override({ jre = result.jre8; })
};
}
- We're using 'result.jre8', which will include any overrides made to 'jre8' (by other overlays, even those being applied after this one!)
- We're using 'snapshot.maven', since using 'result.maven' would cause an infinite loop (since 'result.maven' depends on the output of this override!)
In that link, they're calling their arguments 'final' and 'prev' instead of 'result' and 'snapshot', but the idea's the same. Most overlays call their arguments 'self' and 'super', but I avoided them in this explanation since those names have specific meanings in other languages (they're literally just variable names in Nix).
In a way the clearer dependency semantics of flakes are are driving the reason they are increasing in adoption so much. In essence they work around the problems introduced by overly clever usage of laziness in a nix package.
I don't consider it an awkward hack; it's a very standard least-fixed-point calculation. Developers these days seem comfortable-enough with 'self' and 'super' in OOP; that fixed-point pattern is essentially the same, but without shoe-horning magic keywords into the language.
.. so the sharedOverlays list has one element, which has our in it, which will have a .lib attribute pointing to lib defined above?
Of course, it was written by a guy who actually works on NixOS...
Someone on the team was mentioning that our build/deploy pipeline was a pain in the butt, and it would really be nice to have a proper CI process implemented.
PHD spoke up saying he could help with that, and then spent the next three months wrestling with Nix. I left the company before I saw how long it actually took him, though.
I saw enough to realize it would likely never be worth my time to use. Sorry Nix.
At this point I don't really know if the juice is worth the squeeze.
The problem with Make (other than the awful syntax!) is that it's not composable. For example, GCC has a Makefile, with an "install" rule; yet my C project's Makefile can't use that as a dependency; e.g.
hello: hello.o installGCC
gcc -o hello hello.o
hello.o: hello.c installGCC
gcc -c hello.c
# Pseudocode to import the 'install' rule (renaming as installGCC), from the
# Makefile in GCC's source (depends on gccSource to fetch it)
@import: gccSource
import(gcc-10.1.0/Makefile, install -> installGCC)
gccSource:
wget 'http://mirror.0xem.ma/gnu/gcc/gcc-10.1.0/gcc-10.1.0.tar.gz'
tar xzf gcc-10.1.0.tar.gz
Nixpkgs (a big collection of definitions for Nix) takes this to its logical conclusion: C programs not only depend on their source, but also on the compiler, the shell used to run the commands, etc. In turn, the compiler depends on its source, and a shell, and a compiler (if it's bootstrapped). The shell depends on its source, and a compiler, and a shell to run the commands, etc. The downloader (usually curl) depends on its source, and a compiler, and a shell, etc.
The recursion is cut-off by some "bootstrap binaries" (which the bootstrappable-builds project is trying to shrink even further http://bootstrappable.org )
With this "composability" problem solved, it turns out there's actually no need for "package managers" at all; or indeed for "installing" anything: each derivation just says what other derivations it depends on. Nix can act like a package manager if you want it to; e.g. its 'nix-env' command manages a "user profile" (a symlink at ~/.nix-profile, which point to the output of a derivation, which itself symlinks to the output of whatever derivations we want "installed"). NixOS works in a similar way, but with a system-wide /run/current-system symlink (plus a massive pile of options to configure things more easily).
Hence the slight complexity of Nix (in fact, most of the complexity is in the libraries, like Nixpkgs) lets us do away with build tools (Make, Ninja, Ant, etc.), language package managers (maven, pip, cargo, npm, cabal, etc.), OS package managers (apt, yum, etc.), configuration managers (chef, ansible, puppet, etc.). Nix's "binary cache" mechanism also lets us do away with binary formats like rpm, dpkg, wheel, jar, etc. in a transparent manner (e.g. we can still patch sources, override compiler flags, etc.; we just get a cache-miss)
Of course, that's the ideal. There's so much code written with Makefiles, pom.xml files, setup.py files, cargo.toml files, etc. that most Nix definitions just add those tools as dependencies, and let them do the building!
Can someone explain to me why I'd use this over say docker, ansible, or even a make file?
BTW I'm no expert in any of the aforementioned, I've had to use docker, was thinking of learning about make files, and wanted to look at ansible, maybe.
For example, applying a Nix configuration is more-or-less guaranteed to be an idempotent operation, while this is not guaranteed at all using Ansible : many Ansible modules are not idempotent at all.
This brings many useful features, for example the ability to rollback to a previous state, or more generally the ability to have some determinism in configuration management of complex systems.
Nix and Docker can be combined together, I have many hosts where some services run in Docker containers, where the Dockerfile is defined in the Nix configuration itself, or generated by composing the upstream Dockerfile with my own configuration values.
Better reproducability by default.
Docker containers usually have drift from `apt update` and that can let the "works on my machine creep in".
You can avoid this in your own Docker containers, but you can't fix the whole ecosystem using that approach.
The nix ecosystem avoids this drift and other sources of repriducability issues like the plague.
If he is very popular Technologies with very common workflows, Nix might not seem worth it since the sharp edges from the lack of reproducibility have been filed down by brute force.
You might find as soon as you need to do something slightly different, or need to get around a bug right now, you'll want Nix.
Next I want to write, "For me, I enjoyed it," but I hesitated initially, because I'm surprised that I'm not more annoyed.
Because he recreates the experience of an average user who comes into nix.
To quote,
--- start quote ---
nix-env invocations are too slow for anyone to reasonably use Nix as a package manager. Full stop. Every. Single. Command. 30 seconds. It was unbearable. I mean, it was nice for writing this blog post – plenty of time to get my thoughts down – but if I were an actual, normal user? Trying to use Nix? I would just stop. My Nix adventure would end before I finished running the first command. Because I would assume it was broken, ctrl-C outta there, and go back to using Homebrew (or pacman, or apt, or whatever).
--- end quote ---
Previously:
This stands true for most of the things in universe for most of the people.
Or maybe "Python venv on massive steroids".
gentoo is not a linux distro either (from a certain point of view), more a meta-distribution, where you use the package manager (portage) to build everything from source.
fun fact: docker uses alpine linux, which for a while was a gentoo build, until it got to a point where it could bootstrap itself.
Nix is often compared to Docker, but Docker is essentially just a way to run programs (in a sandbox). Saying "I'll build it with Docker" says how you're going to run it, but not what you're going to run. Unfortunately, the "what" usually turns out to be a shell script, which runs one package manager to fetch another package manager, downloads a bunch of files from some random URLs, etc.
Nix is all about keeping careful track of the dependencies between things, such that all the fetching has been done by the time our scripts start (access to the network, and any paths outside the build sandbox, is disabled by default).
Also, the way Docker tends to be used involves downloading an entire Linux distro, just to run those scripts. Nix does the opposite: scripts only have access to precisely what they've asked for.