Hacker News Reader

Comment by jcranberry | Hacker News Reader

It's been a while since I used NixOS, but I never really had an issue with the language. What was frustrating was in package solutions there were all these special functions which as far as I can tell were undocumented and I couldn't figure out what they could do. So there was sort of this gap in terms of documentation between theoretical understanding and tooling/functional understanding which I struggled to cross.

View on HN

As someone 8 months into a major Nix packaging effort (1200 package definitions, mostly auto-generated), this is 1000% the most significant issue. Reading the pills can get you the first little bit, like the difference between derivation and mkDerivation, but so much later stuff is just completely undiscoverable and inconsistent, like the fact that overrideAttrs may subtly not do what you want if it's a Python package since you have to use overridePythonAttrs instead, but the corresponding function for other builders (eg overrideBazelAttrs) doesn't exist.

This is indeed not very good. Note there is a reason that overridePythonAttrs exists: it overrides the call to buildPythonPackage instead of mkDerivation. There is/was an RFC on standardizing overriding in Nixpkgs but it got stuck. I think for these things to improve what is really necessary is funding to improve Nixpkgs. These kind of issues are fairly hard to solve as they span multiple ecosystems and require coordination.

Oh, of course, and in my case it was an important discovery, because my custom packages were all also defined in terms of my own mkDerivation-wrapping function, so I happily copy-pasted the definition of overridePythonAttrs to make my own overrideMyAttrs version of it.

But it was brutal that I had to just stumble across this. I think the article correctly identifies that there is decent enough documentation in Nix for:

- Beginners to the nix language/concept (Nix Pills)

- Beginner users of NixOS (the NixOS manual)

- Beginners to packaging for Nix (wiki articles, blogs)

But there is a huge, huge gap for the user who is trying to cross the chasm and become a true expert in it all— who wants to understand at a deep level things like how overlays and namespaces work, how bootstrapping works and what is the division of responsibilities between nix and nixpkgs, plus the inner workings of things like buildEnv and stdenv.

I mentioned it already below, but just to emphasize: I think Burke Libbeys playlist on Nix [1] is that missing piece. It lays down a solid foundation of understanding on what Nix is both conceptually and syntactically. You are then able to derive the rest of the needed understanding by clicking through the nix source of whatever it is you are trying to figure out.

[1] - https://www.youtube.com/playlist?list=PLRGI9KQ3_HP_OFRG6R-p4...

As someone who really likes the idea of declarative system configuration and management, shares the vision of nix and guix, and have been dipping my toes by using nix package management on macos (and sharing most of the experiences mentioned upthread):

I do thank you for the pointer, and I think I've come across these videos before, _but_ to me, video is probably the last format I can assimilate this type of information. Usually, in my experience, well designed systems have a way of making sense so that documentation and understanding flows naturally from base abstractions, and this is something I miss in nix. I would like to try guix to see whether that makes more sense to me, but I am hesitant given that nix seems to have so much more of the mindshare.

One day I'll try to give the video tutorials a chance, maybe it will clear things up. Thanks again and thanks to mr Libbey as well.

> well designed systems have a way of making sense so that documentation and understanding flows naturally from base abstractions

Honestly, thats the main revelation I had when going through that playlist. Basically Nix is just attribute sets (which is sort of just a JSON object). The entire thing. Nix files are just some preamble and then they return an attribute set at the end. Understanding that will get you 90% of the way there in terms of understanding the syntax and being able to cut through the "weirdness".

At that point its about understanding actual concepts like overlays, derivations, etc but its much easier now that you understand the attrset concept.

I hear you, I think I understand what you mean and I definitely want to believe you, but how come this can't be explained in a succinct written documentation, instead of a series of video tutorials by example?

I do realize it's a work of passion done by numerous volunteers, to whom I feel gratitude, but at this point, why can't this be adequately defined, in a written form, in a way that is understandable? To me, the concept of a series of projections of declarative configuration statements onto a final state is not hard to understand in itself; yet, I am somewhat struggling to create and maintain such a configuration without feeling that I am just copypasting bits and pieces from random people.

I guess I just have to bite the apple and spend a few days watching YouTube to see if I reach enlightenment - at this point, I barely understand the function names.

Yeah like I said, I struggled for a couple _years_ on having a feeling of complete confusion when copying a shell.nix to every project I spun up. I was never able to understand _any_ of the written documentation. After that playlist I easily click through the actual nixpkgs repo itself and find almost everything I need to handroll my own derivations.

It seems like the community is just a bunch of people who have had that lightbulb go on already, so its hard to go back and view things without the lightbulb on.

The language reference worked out fine for me but the chasm between "basically running" and "knowing what I'm actually doing" has been substantially painful and I'm still not on the other side.

Though I say this while continuing to slog through said pain as I find time because once I've learned a particular thing it's always felt like it was worth the effort to get there.

100% this. The language is designed for its use case which is packaging and configuration (nothing more or less). It has a learning curve due to being lazy and functional but works great once you get the hang of it. But the documentation of all its functions is so annoying. You have builtins and the nixpkgs functions[1]. There is learning the language, and then learning how to use it. Then there is the entire ecosystem of custom packaging functions that have their own pros/cons [2]. The issue isn’t with the language but the difficulty with trying to make existing tooling work the Nix way. That part is where I agree with the curse of nix. But the effort is worth it because once the packaging is complete it just works (forever).

1: Best resource I’ve found is this: https://teu5us.github.io/nix-lib.html

2: The status of lang2nix: https://discourse.nixos.org/t/status-of-lang2nix-approaches/...

However, until this works, nixpkgs should provide a wrapper around FHSUserEnv which allows developers to develop without the curse.

Convenient (version-addressed!) FHSUserEnv is exactly what I want out of Nix. Land me in an environment that has a list of deps (at specific versions, not hashes!), let me go mess with it.

These look pretty promising. Maybe I'll give nixOS another shot because I really was a big fan.

The best way I’ve found around this is to always keep a local clone of nixpkgs so that I can grep for where functions are defined and hope like hell that there are some comments in the vicinity. Definitely not ideal.

If you're using Nix on your system, you can also find nixpkgs symlinked under ~/.nix-defexpr/channels/nixpkgs/pkgs/ or so.

And if you have flakes and nix-command enabled, you can run `nix edit nixpkgs#lib.sort` to get the definition of the sort function.

(This will just tell you it's imported from lists. `nix edit nixpkgs#lib.lists.sort` will get you the actual definition.)

Nice, I did not know this. Thanks!

What really helped me were the videos of Jon Ringer: https://www.youtube.com/user/elitespartan117j27

Glad someone found them useful :)

I cannot thank you enough for these. It is so much useful to see someone doing a task.

I believe what Nix really, really needs. Is to separate all the lib, builders and frameworks for languages need to be separated out of it (probably into separate repo, maybe even repos), and treat it like a stdlib is treated in any other language.

That means, it can't be just modified by anyone on a whim, all of changes need to be done through proper RFC mechanism. And anything in it must be documented.

Having all that done would greatly improve learning time, it would probably make it easier to maintain other packages as well.

It made sense that it was done this way originally, because initially no one knows what would be needed, but right now many things are solidified and don't change that much.

Fully agree. It is not the language, but the functions around nixpkgs.

Reply on news.ycombinator.com