How Nix and NixOS get so close to perfect
christine.website
christine.website
And I don't mean a lack of documentation — what I mean is that the obvious decisions that have been taken (naming everything "Nix", using Haskell as a base for the syntax, ...) don't really fill me with confidence that in other spots decisions have been made in a logical and/or straightforward way.
I am trying to have a less terrible experience with dependency and configuration management — but what I am not trying is making dependency and configuration managment my nerdy hobby. I have enough nerdy hobbies already.
So in short: My worry is that the seemingly big investment in terms of time is not worth the advantage this might give me over e.g. a traditional debian system.
The thing is: I really, really want an system that does the things nix does, but nix doesn't seem to be it. I am grateful about the honest presentation, but if nix is really the future, these paper cuts have to be dealt with.
I agree that the name is bad and has always been bad. I wonder what they were thinking. But the syntax of the Nix language is not based on Haskell. It would be better if it were, but the Nix syntax is actually based on trying to twist a functional language into looking like a Unix-style configuration file. It's horrible, and conceptually big details like "this is a function" are hidden by subtle syntax (a single colon). The liberal use of semicolons and the use of space-separated lists is another concession to looking like a config file. I feel Nix would have been better if it didn't try to cater to older conventions in this way. Sometimes the old ways are just bad. Incidentally, one of the big advantages of Guix (a Nix fork/derivative) is that it uses Scheme as its declaration/configuration language.
I switched to Nix some years ago on my desktop system. While I was initially quite frustrated at lots of the paper cuts - particularly the byzantine design of Nixpkgs itself which is built around manual programming with fixed-point combinators - I was carried through by two things:
1) I was intellectually convinced that the basic premise was sound.
2) Getting a basic desktop system running doesn't require you to understand all the complexity. NixOS works really well out of the box with its standard settings, and making basic configuration changes and adding/removing packages doesn't require you to know anything about the Nix language or the design of Nixpkgs.
Not much later I was able to do pretty radical things like changing the version of LLVM used by Mesa on my system, to work around a defect in AMDs GPU drivers[0]. I have no idea how I would do something like that on Debian. Even better, when this workaround later became unnecessary, I just removed the pertinent parts from my configuration file, and it was like it was never there. My system is fully declarative and not soiled by the remnants of previous hacks.
[0]: https://github.com/athas/dotfiles/blob/d495aeb85fe38569eb212...
I don’t really think this is a problem at all? I mean, at the particular design point the language occupies there are basically three possibilities:
- Kind of like Lisp (parens everywhere, painful with pervasive currying);
- Kind of like ML (var and fun everywhere);
- Kind of like Haskell.
Nix is chooses the last of those, and it does not have most of Haskell’s funkiness: no “alignment”-based blocks, no custom operators, no separate type sublanguage. The merged string interpolation / field lookup thing looks kind of funny, but has nothing to do with Haskell. The scoping rules for 'with' are weird, but, first, are not syntax, and second, have nothing to do with Haskell either.
Really, if one were to design the language that fits Nix’s description—a dynamically-typed (not Dhall), lazy (not Guix) higher-order (not CUE) language with a JSON data model—I don’t see how it would look much different. It could be less consistent, it could have more infix operators for things, or it could use a list concatenation operator that is not like anything instead of like Haskell. That’s it? If anything, any alternatives I can think of would be less conventional and/or more Haskell-like.
How so? Where's the Java/Python-like languages in this consideration, why wouldn't those work? Or something more like Ansible?
I'm pretty sure that you can express the same concepts for the most part in something imperative as well.
How nix is architectured is exactly why it achieves purity and reproducibility, not the configuration language. It isn't even a goal of ansible in the same sense, per design.
The building blocks of nix is simple, of course it can be implemented in something imperative.
The fundamental design of Ansible is that you do imperative steps to reach an end state. The fundamental design of NixOS is that you describe the end state and the compiler derives the intermediate steps.
Yes, normally things work out with Ansible, but the devil is in the details and can involve a lot of fighting configurations that are non-cromulent.
And the problems with ansible would not be any better off if they had used something like the nix-language instead.
No, the fundamental design of configuration management systems (Ansible, Puppet, Chef, Salt, CFEngine) is that they are declarative. You say "this software should be installed, this user should exist, these files should have these contents and permissions, etc" and they figure out the right steps to take on your OS to move from the current state to your desired end state.
You are right though that they can produce unexpected results since they do not describe the entire system from the ground up.
You are right that imperative definition is not a problem for reproduceability, but it does make things harder to reason about since you have to mentally keep track of state between steps.
Chef doesn't allow you to specify the end state. Instead it allows you to specify a list of actions to be taken during deployment. The end result is affected by bunch of external inputs sent from the Chef server and every single bit of state the target server is able to observe. It is therefore impossible to reliably determine the end state of the target server before deployment. Our team has been burnt by this more than once.
Other similar systems are more or less the same. They have every hallmarks of being an imperative system.
def packageInstaller(pkgs: Pkg) -> Derivation:
x = {
req: [pkgs.a, pkgs.b]
build:
...
}
x[req].append(pkgs.c)
return buildDerivation(x)
Keep in mind that mutability and immutability share a certain isomorphism. Within the nix expression language you often do something similar to mutation just as any other functional language. For example:
listVersion3 = listVersion1 ++ AdditionalStuff
What's great about Nix the language is that its design matches beautifully with its purpose. Nixpkgs is a single giant program, the output of which is a set of 70,000 package definitions. Since Nix uses lazy evaluation, you can specify one of the attributes of that set, a single package, and the Nix interpreter will evaluate just that one package expression. Of course, if that package depends on other packages, those will get evaluated as well, because they're needed as inputs to the package you care about. So package dependencies emerge naturally as computational dependencies between function invocations. This is really powerful, and over the years, Nixpkgs has evolved more and more sophisticated ways of defining packages - package overrides, the overlay system, and most recently, flakes.
So yes, using an imperative language would technically be possible, but I doubt that you could use it to build the largest and freshest package repository with a fraction of the contributors that other packaging systems have.
And honestly, the package situation is the biggest downfall of nix(os), there are tons of packages, yes. But they are often buggy (simply because nix is so different), not maintained (understandable, people move on), or plainly missing (because the few maintainers don't cover even popular niches). Of course that is going to be the case for a new system, but I'm flabbergasted every time it is being touted as an advantage of nix(os). And I feel that it is quite counter-productive tricking people into the ecosystem and giving the impression that it is on par or even better than any mainstream OS. It is making great progress, but it has a long way to go.
I have a big problem with "it has a long way to go" arguments. "It has a long way to go" in comparison to what? Every project I know of could be described as having "a long way to go".
If we're comparing it with ubuntu/debian, there are few packages that ubuntu/debian have that nixpkgs is missing. On the other hand, what's the status of getting kubernetes in ubuntu/debian? Last I heard it was mired in https://lwn.net/Articles/835599/. And what happens if you use the ubuntu/debian prometheus package? You currently get a version that many people would consider "too ancient to be useful". I could easily use these points to argue that debian has "a long way to go".
The real answer is "they're different". It's a mistake to think the two are just trying to replicate each other piece for piece.
The same goes for people arguing that the linux desktop has "a long way to go", whether they were making that argument in 2001 or 2021.
In comparison to any other mainstream OS nixos is going to have a lot of friction for most usecases. That is what I mean with it having long ways to go.
Part of that is configuration.
Other part of it is the package system being very immature. It is a direct consequence of the massive task nix has set out for itself, but that isn't always comforting for the end-user. And oh boy, the situation with looking for github-issues for packages is a freaking nightmare.
Just because nixos has a package for it doesn't mean that it does what it should or is up to date. That of course isn't the case for any package management system. But, all other mainstream OSes have matured. Nixos has a long way to go.
I love nixos, it is my main driver on my laptop and I run many VMs with it. If the future isn't incorporating the selling points of nixos I'm going to be dissapointed. But unless you want your OS to be your hobby I most certainly would not recommend nixos.
Nixops is another things that disappoint me / has long ways to go. It is the logical extension of what nixos is and it isn't really mature. There are lots of competing efforts for it and as someone willing to really invest time in it I just get exhausted.
I would love for a distro with declarative configuration management. But I don’t have time for more hobbies right now.
This is the first time I would have heard that point of view, could you elaborate?
This seems to be the argument that comes up over and over with go, JavaScript, Ansible, etc. Sure one can take an imperative language and try to follow FP principles to emulate good things. Then one is comfortable with how it works while not really understanding FP and when it trashes the system by being implemented as a sequence of trashes of the state it is just a learning experience of user error.
If you make it more like Ansible, you will end up with ... Ansible.
If you want to use Ansible you are perfectly welcome to, but as a Nix person the entire Ansible way looks like a completely wrongheaded approach to system management.
Ansible is “bottom-up” (you start in the middle of a server’s lifetime without knowing everything); many more cloudy solutions like Kubernetes assume that a bunch of infrastructure is already set up; container registries, build servers, and the control plane.
My first Ansible use-case was to install K3s on a dedicated machine where I could only choose the OS image from a few options like Debian and CentOS.
I would probably prefer Lua to Python because while Python can sort of do declarative EDSLs they invariably suck, and the expressive power and general style of the two is largely similar; Java has all those problems times a hundred so I expect you would end up staging the whole thing into a config processor in Java and a config in something else; but while the ergonomics may vary it’s certainly not impossible. Guix, Pacman with Aconfmgr, or the venerable GNU Stow are all this to some degree.
That is not what I was saying, though. The semantics of the Nix language (JSON-like data model, lazy, higher-order, dynamic types) are a vital part of the implementation approach taken by the Nix package manager, and once those semantics are fixed the syntax (so I argued) has to be broadly similar to what it is now. (You could force a Java- or Python-like syntax upon an ML- or Scheme-like language if you really wanted, but the result would just be miserable to program in; the F# people tried but mostly gave up.)
Is that approach the right one? I suspect so, but it is still to early in the life of the whole idea to be really sure. The idea that a dynamically-typed lazy (thus pure) higher-order language is a good middle ground between a completely declarative configuration and a completely imperative setup script (neither of which can usually stand alone) is a fresh and interesting one, and I’d very much like to see it explored. I’ll be the first to admit that the documentation on Nix internals is lacking, but once the requisite source spelunking is done it’s delightful how the whole thing comes together and how much flexibility being written in itself affords it. (Did you know that NixOS was originally a fun experimental addon to the whole thing? The packaging language accidentally ended up being powerful enough to make a usable and novel system configuration language.)
But the complaint seemed to be that the syntax (which is relatively superficial and does not require particularly deep argumentation) is Haskell-like, not that the implementation approach (which is crucial to the whole design and requires arguments on the scale of a research paper) is wrong. My point was that the latter pretty much implies the former.
Huh?
The Nixpkgs repository has over 80,000 packages [1] contained in a single key-value store. The design of the Nix language not only makes this sustainable, but even compelling. For one thing, Nix package descriptions are composable and easy to manipulate programmatically. It's common practice to create new packages on the fly out of existing package definitions, and it's what the Nix language is particularly good at. Things like this add a lot of complex inter-dependencies within the Nixpkgs code base, and it still manages to be comprehensible.
The same approach isn't practical with general purpose imperative programming languages. A quick glance at Nixpkgs repository reveals that it contains over 2.4 million lines of Nix code. Imagine that much Java/Python code manipulating a single global mutable data structure. It can only result in a nightmare.
Additionally, the notion that the Nix is Haskell and is therefore incomprehensible is an often repeated meme without any backing details and needs to stop. If any language is similar to Nix, it's Jsonnet. But I haven't seen a single person criticizing Jsonnet [2] for being Haskell. If you can understand JSON and have no problem writing functions for any common language, then there really should be no problem learning the Nix language.
[1]: https://search.nixos.org/packages [2]: https://jsonnet.org
{
"foo": "bar",
"COMMENT": "This bars the foo"
}It's not really 'kind of like' any of Lisp/ML/Haskell IMO, and a good fit I think. Maybe it wouldn't allow everything or easily, but then if we're already only talking about being 'kind of like' (not actually using) an existing language that's not a problem.
(It's more like Prolog I suppose, not syntactically but in declarativeness, which is also missing from your list. Unless it's lumped in with Lisp for having too many parens.)
I didn’t think HCL had laziness (which is vital to how NixOS module or Nixpkgs overlay A can set or use things in B which in turn set or use things in A), I’ll have to look at the theory more closely, thanks. (Its documentation tries pretty hard to make HCL look boring and weak, I guess it’s my fault for believing it.) Doing logic-programming-like things would probably also work as an approach to achieving these goals, yes, just a different one from that implied by my constraints above. I’d count CUE as being interesting in that direction as well even if it’s not technically logic programming (is it?.. the bibliography is not great). I’ll guess we’ll see.
> I don’t really think this is a problem at all?
Of course it's a problem. You shouldn't use a syntax that has been soundly rejected by the industry.
That's irrelevant - if you want uptake, you need to support what user's want. It's been clear for at least a decade now that programmers don't want Haskell syntax.
Whether it's a good syntax or not is irrelevant.
Could very well be - either way the language has been rejected. Boasting of Haskell characteristics[1] is, IMO, a great way to turn off prospective users.
[1] I don't really know if the Nix project actually does blog posts and reachouts where they boast of Haskell-ish similarities.
Before the iPhone no one knew they wanted a good camera and "apps" on their phone.
See the important point here -> "knew they wanted"
> Before the iPhone no one knew they wanted a good camera and "apps" on their phone.
> See the important point here -> "knew they wanted"
Haskell has been around now for something like 25 years. You'd have a point if the iPhone was unpopular for 25 years before users finally decided to like it.
It wasn't. Users saw the iPhone and wanted it the minute they saw it. Programmers have been looking at Haskell for 25 years and rejecting it.
I really don't know why you think that true:
https://wiki.haskell.org/Haskell_in_industry
And i really don't know why you speak for "programmers", just speak for yourself...maybe?
> I really don't know why you think that true:
Because it doesn't show up any any metric as having significant usage. A list of 100 companies using it doesn't make the hundreds of millions of other companies not using it go away.
FWI HN was not written in JavaScript or PHP but you can lookup that for yourself.
Nowhere did I claim that - you're reading things I did not say.
Claiming that something is rejected by everyone does not make any statement on the quality of that thing.
[EDIT: It's kinda common knowledge that Haskell is barely used in industry. I cannot remember coming across a single non-academic programmer in the workplace who actually wanted to use it, other than myself]
It should be clear now, that this is a wrong statement.
> It should be clear now, that this is a wrong statement.
You're correct - it's rejected by roughly 999 out of every thousand programmers.
I literally claimed otherwise, elsethread
> It's a reflection of what the programmers want. Apparently they wanted syntax with braces and semicolons.
Javascript, C, C++, C#, Python and Java have seen and are still seeing heavy use in the industry. Go has gotten traction too, now.
Any language that can be seen on a common metric to be more than low single-digits (and, in some cases, fractions) of a percentage can be safely said to be "used in industry". I cannot think of any survey amongst programmers which shows Haskell usage outside of 1%-3%.
Haskell's slogan is "avoid success at all costs". Haskell isn't aiming for mass adoption. It wants to be a language for researching functional programming, it just so happens it's a useful language, and people write real software with it too.
From https://haskell.foundation/whitepaper/ > Haskell's slogan of "avoid success at all costs" was a clever and cheeky way of saying that innovation and research in programming languages, especially in functional programming, needed some insulation to succeed
Nix doesn't have such a slogan as far as I know. It, however, started as a thesis to try and research and innovate new ideas. https://edolstra.github.io/pubs/phd-thesis.pdf The Purely Functional Software Deployment Model.
Twenty years later, it's been rather successful as a research project, one of the most active repos on GitHub. It has gained a lot of usage and support without trying to be successful, popular or market itself.
The Nix language has helped it achieve what it set out to achieve. It's appealing to people who are interested in purely functional software. If that doesn't appeal to Javascript / PHP devs that's ok as it isn't trying to appeal to them.
The Nix expression language is small and pretty simple; everything is an expression.
The difficult bit is there are no javadoc/jsdoc/pydoc type docs beyond https://nixos.org/manual/nix/stable/expressions/builtins.htm... where you can go search easily. You have to go through the source code to find functions and see how they work. Also, you have to accept Nix is a ML style language, not a C style language. It's different to what you may know, but that isn't a fault of the language if someone only knows C style languages. The language is as it needed to be like that to achieve the design it wanted. Personally, I hate using Ansible firstly because of YAML. Secondly, Ansible is a task runner, often feeling like you are over abusing Ansible for things that are not a natural fit, and the debug headache dealing with determinism by convention only.
I think it comes down to Nix isn't for you. That's ok.
This isn't well-known - read through this thread - their are many people in this thread who deny that Haskell usage in industry is insignificantly small.
If you want to argue against the semantics, by all means, but “industry doesn’t think it looks pretty” does not cut it, even “I don’t think it looks pretty” would afford more respect on my part (not least because the speaker is then able to explain what specifically they find off-putting or annoying, while “the industry” isn’t).
More details on syntax vs semantics in sibling comment[1].
I'm not arguing against semantics, and "the industry doesn't think it looks pretty" is not my argument either.
My argument is that, if you want the industry to accept something, it needs to be acceptable to the industry. Haskell-ish syntax[1] has clearly been unacceptable to the industry for a long time.
It's been rejected, again and again. At this point, it is pretty much guaranteed that attaching Haskell syntax to your new config format/language/application/whatever is a recipe for failure.
It's not that the industry just thinks it looks ugly, it's that the industry has outright rejected how it looks.
> even “I don’t think it looks pretty” would afford more respect on my part
My opinion on Haskell syntax, and whether or not you respect my opinion, is also pretty much irrelevant if the industry has already rejected it. I could be thinking that it's the best thing since sliced Jesus, but that won't change the fact that the syntax of Haskell has been rejected by the industry.
[1] Amongst others.
What?? Galois, Facebook, IBM, Twitter, AT&T, Bank of America, Barclays Capital, NVIDIA, Microsoft and GitHub use Haskell.
So? The usage of Haskell within the entire industry is so small it's not even a rounding error.
It's hard to make the argument that the industry has accepted it when the industry usage is too tiny to even measure properly.
Programmers, and the work they do.
> and explain what credentials you have to know what "they" want.
It sounds ridiculous that you need "credentials" before you'd believe that Javascript, Python, Java or C# programmers, programs, projects and work doesn't each outnumber Haskell roughly 100 to 1.
Just about any metric you choose to use to evaluate what the industry uses, Haskell has so little usage that it's literally difficult to measure.
Job postings, TIOBE, RedMonk, SO survey, google-fight, you name it - Haskell is nowhere near rounding error percentages.
Maybe you should ask yourself why it doesn't show up in significant numbers on any metric rather than asking people what credentials they have.
As far as I can tell, the only languages with Haskell-like syntax that have tried for serious industry adoption are Haskell, Scala, and perhaps Elm. (Idris is a borderline case, but it’s young and in any case I think we’re still a couple generations away from viable total languages.) Each of these has unconventional and similar semantics, different but complex type systems, and is trying for general applicability, while Nix has somewhat different semantics and no static type system at all, and is also trying to cover a very limited application area. Even Nix’s syntax, to my eye, is about as ML-like as it is Haskell-like.
How can we be so sure it’s the syntax, specifically, that is the problem? Are we controlling for these differences in any way at all?
It would be another matter if F# or OCaml or even Dylan was a runaway success while Haskell was not; while strict vs lazy is a hell of a difference I could be convinced that the syntax is the problem if people from the industry tried one after the other and consistently backed off in disgust, but that is not happening as far as I can tell.
I really don’t see how the verdict here could be anything but a definite “inconclusive”.
Finally, I’m also not seeing why I want industry acceptance. (I don’t think Xe’s post discusses it at all.) I want to use it, I want other users to discuss it with, I want it not to die a slow unmaintained death in five to ten years, but otherwise... Meh?
Advocating it to a single organization seems like a limited task (which I might fail), but advocating it to “the industry” seems like arguing with a force of nature (and I will fail). There would be some advantages to me if it were adopted, but then again it’s a first-generation system that has already accumulated substantial legacy cruft, so maybe I don’t want it to be adopted as it is now? Exploring the solution space? Definitely. Adoption at this particular stage? Not really sure, hard to get worked up about it.
P.S.
>> even “I don’t think it looks pretty” would afford more respect on my part
> My opinion on Haskell syntax, and whether or not you respect my opinion, is also pretty much irrelevant [...]
Wrong kind of “respect”, sorry: not how much I trust it, but how much concrete information I can expect to squeeze out of it (complete trust helpful, but neither necessary not sufficient). Opinions of large groups of people, even if reliably known, are notoriously hard to turn into actionable insights.
That same industry which embraced PHP and JavaScript? Is that an authority, on what?
Also, the industry hasn't rejected Haskell, it has just been learning from it very slowly. My prediction is this trend will consistently continue at the same glacial speed.
It's a reflection of what the programmers want. Apparently they wanted syntax with braces and semicolons.
Someone else pointed out elsethread that maybe it's not the syntax, but the semantics, that is being rejected here.
This is such a weird thing that keeps coming up. Why would the language have to be lazy? Obviously, there are tons of things that are lazily evaluated in Guix, e.g. the whole package graph. Why would Scheme have to be a lazy-by-default language? It baffles me that this objection keeps ending up on people's lists.
But VHS is apt, in the way that matters most: the number of apt package maintainers is staggeringly larger than even both combined. More maintainers mean two things:
1. If you want to install something, it has a one-liner install with much higher probability.
2. If you have a bug, or the package has a security flaw fixed upstream, updating your system will yield the version that fixes that with much higher likelihood.
I try to use Guix alongside APT (because using only Guix is tying two hands behind your back). But:
• The shared objects management (libc and the like) is a pain with no guidance; APT binaries expect the .so somewhere, while Guix binaries expect it somewhere else.
• What remains as usable are self-contained binaries, which are frequently severely out of date. For instance, I had installed the fish shell with Guix, but by now an APT-installed binary that detects fish assumes a more recent version than what Guix offers, causing errors. The Guix community cannot figure out why upgrading the fish package trips up some tests fast enough to keep up with APT: https://issues.guix.gnu.org/51064
Guix is certainly not favoured by those who see themselves as leading the "project".
GNU is hardly even a project in the traditional sense.
Guix on the other hand is.
Nix/NixOS is kinda shit, but at least you can easily run closed source software on it.
"Use case -> how to achieve it", more than "functionality -> how to use it". Nixpills seems to flirt with both concepts, but also is a bit... showing off?
Just plain "I've got this app, how do I package it", "how do I overlay it", "why are there 5 ways of doing overlays that people use differently", "what is the actual expected signature of various flake elements", "what packaging helpers are available and how do I find them", ... would be amazing.
I have issues with nix docs trying to be clever. For example https://nixos.wiki/wiki/Overlays - "Careful, we're using a different self and super here!" - ok, then name them something more descriptive!
https://documentation.divio.com/
The big idea is that it separates types of documentation by its usefulness to a particular user that is approaching the docs with a clear goal. Here the goals are loosely organized along two 'axes': working-studying and theoretical-practical. Personally I think the model is missing a user-developer axis (especially for a developer tool like nix), but overall it's a great lens to view any documentation project through.
On average I would say it takes maybe a couple months of wrestling with it to fully pick it up and you'll still encounter problems on occasion. But the end result is like riding a bike. Once you know it... you know it.
It may seem a little offensive but it is very much true. It's a function of time and intelligence. I do not consider myself a person where my intelligence is high enough that I would pick it up as the time and effort for me doesn't justify the reward. Every programmer on HN likes to think of themselves as super smart, but this just isn't the case.
The only reason why I learned it was because my Job required it. I now use it as a daily driver but if it wasn't for the job, it wouldn't have been worth it.
For most of us, it's a benefit/cost analysis. I've no doubt I can figure it out, but last time I tried, a package came with defaults that differed from just running a .jar file. I saw no way to change that config anywhere. Things like that make it not worthwhile unless you're doing something that makes it worthwhile.
I think docker / K8s probably fits better in those usecases anyways, though I do use nix / guix at times for packages that work properly.
I think requirements and plain ol' luck (or never needing anything fancy) can play a bigger role, for those of us who sometimes distro-hop a bit.
I use NixOS on my 2 hosts at home. It took about 3-months of tinkering with it and testing different things until I was comfortable to do that. But the headaches it’s solved make that investment more than worth it. Declarative config and reproducible builds are just amazing, I could never go back to a regular linux now.
However it takes about 6 months for a new hire to ramp up to nixos. The only reason why our company uses nixos is because someone really smart was part of the founding team.
Docker is a good alternative... but honestly if you think about it, instead of using containers for OS's an OS itself should be designed to deal with this problem. It works in the same way type script works as type script is an effective an elegant hack to get around the warts of javascript.
The Nix language, however, is about what it needs to be. I don't like it, mind you, but I think it strikes the right balance between config format and programming language. There are things I'd tweak, but it's pretty much what it needs to be. Any similarity to Haskell is mostly coincidental. Because of that, it's also not especially hard to learn. If you'd like to give Nix a shot, I think you'll find that it doesn't need to be your hobby to be useful quickly.
The post hits the nail on the head: The documentation is horrible, and barely differentiates between the language, the package manager and the OS itself.
I really like the idea behind nix; maybe something with better execution comes up one day :)
It's not "getting so close to perfect". A jet-powered cart (from the comic in the presentation) is not "close to perfect" by any imaginable criteria. Well, unless your idea of perfect is "a jet-powered cart".
In particular, Nix can be used in a non-invasive add-on manner on ordinary distributions. We have it installed on RHEL servers at work, where it is used exclusively as a build system and for "userspace programs" (not in the kernel sense, but in the sense of programs that are not core to the functioning of the system).
In a world where there are thousands of things for me to learn (many of them with poor documentation), Nix is just one such thing.
The question is, do I care about its purported benefits, and sink my time and effort into it, or should I sink time and effort into learning Unreal Engine, playing Rimworld, or going for a walk?
So far, for me (and most of the world) the last three (and the rest of the thousands of things) win over Nix.
As a diehard Emacser, Emacs has been waiting about thirty years for someone to make it a prettier wrapper.
It's possible to be so power-user-friendly that in practice no one wants to make it end-user friendly.
It is the way I’ve always wanted a package manager to work, and it really does work. It feels insane to use anything else now, I can’t go back.
No, not really. Tech people especially are susceptible to sunk cost fallacies We also get the kick out of conquering complexity.
Is truly great.
About complains:
- Nix as "language": Is stupid, that is true. As any other stupid language on this space, name it: bash (TERRIBLE), docker, yamls, pseudo-TOMLs, etc. So is not that worse than the median. I wanna know what other real option is here, except use a language as python instead.
- Documentation: Is not lack of it, is how spread and badly stitcher together. is like docs made by JavaDoc. But I remember Docker at first, and I waste as much googling as with nix. Docker is not intuitive at all, and worse, it change per distro!
The actual thing is easy, and the best thing is just look at the code. Most of it is not that hard to figure than docker files, yamls, etc.
What was very confusing is the idea of how each config are done. Exist many things that are not "together" like setup NGINX also need to touch the firewall, then maybe systemd.
Is not different to deal with unix, that is a mess, is that at first you think all is "just work" by default.
More exactly: Not exist a clear convention that is applied everywhere. Lack of uniformity is the main trouble. Still, same as other tools in this space.
Also: Very hard to debug things.
Also 2: The hack around "home manager" to manage user config is a hack. Make not sense: I use too much time trying to do it "natively". Do it manually in *nix is so simple "just cat/touch this file in this path", so is the main reason I almost drop it.
---
The whole migration for me take 1 week. That is not that much than before the first time I doit with custom docker files (One of the companies I work for before I use 2 weeks making it. This was when alpine/small containers were not as common as today).
So, YMMV but the end-results is very nice.
I understand it allows for stable binaries, without dependency hell, easily installable, possibly multiple versions of the same package; and for easy rollback of the installations. Because code modules are versioned by hashes of their full dependency tree and build options.
But configuration files?
I once installed the Nix package manager on my Ubuntu, and installed some apps with it. Nothing worked. Because some files I have in /etc/ were not compatible with Nix versions of the applications. Those configs were about some deep mechanics of X-related libraries. I never edited them and I don't understand their function.
So, it was impossible to use Nix packages side by side with my Ubuntu. And I suspect the same can happen on plain Nix OS with multiple versions of one package.
Probably I am missing something, but that does not look perfect.
1. For per-user configuration there is home-manager which symlinks dotfiles to Nix store based on a single "home.nix" derivation (state). Home-manager is usable outside of NixOS and, in my opinion, it is better than most dotfiles managers. https://github.com/nix-community/home-manager
2. In NixOS /etc is composed in the same way from the system derivation.
3. Outside of NixOS, I think, one can build systemd units which refer directly to configuration files in Nix store. Probably one can add config paths into wrappers, but it will be very fragile and won't scale.
To me personally it doesn't really have much appeal.. Being able to replicate my desktop from scratch is not something I need often. And when I need to make a config change I don't want to bundle that with a package upgrade.
Even for my servers I don't subscribe to the "cattle not pets" ideology.
I just mean to say it's close to perfect for its target userbase. But different ones exist and this seems to often be forgotten by Nix evangelists. It's a good tool for some jobs but not for all.
I must say that using nix on my primary workstation makes it feel like an appliance. The system generation roll-back have saved me a lot of times when an application update brings a regression.
Also I love the ability to mix packages with source compiled software, which is one of the reasons I use FreeBSD on my primary desktop. It does that combo really well with its ports collection.
Many applications don't obey it though. Makes sense because they'd have to make code specifically for FreeBSD and its marketshare is tiny.
> it's basically the same kind of technology as Ansible or the many alternatives It's not Ansible because Ansible cannot guarantee state convergence.
Nix fills a void in package management. It is like Ansible or Terraform for a semi-idempotent distro. It is probably quicker in most ways than building something similar with other configuration management tools, but I still prefer other tools because the language for Nix is just bleh. Had they went with a simple YAML format or HCL, then I'd give it a chance, but because they didn't I'm certain eventually someone will create something better learning from the mistakes of Nix.
Honestly after a while the language starts to fade into the background and you just focus on what you are doing. Keep in mind that I am an expert at this point and have prior experience with Lisp, Haskell and a little bit of ML; so the overall flow of the configuration language and the like is easy for me to feel my way into. I have been trying to work on documentation to help get people up to that level; but trying to explain things you understand intuitively is hard :)
I'm trying though. With my Nix flakes series I'm probably going to rebase a lot of it around using NixOS with WSL to make it easier to approach for newcomers.
"Like" does a lot of work here. Those systems go one way: what if we applied this concept to managing existing systems. Nix goes the other: what if we built a system around this concept. I think that's a nice elevator pitch, but misses a lot otherwise.
This is what I'm referring to. There isn't anything currently as far as I'm aware that takes it as far as Nix. Sure, Arch has archinstall and there is also cloud-init.
There are examples of using JSON to help manage that:
https://github.com/archlinux/archinstall/blob/master/example...
But, ultimately I believe a lightweight Go/Rust tool that takes a YAML file and does essentially the same thing will prevail.
Compared to Nix, every other package manager is a security liability.
Nix is great, but I don't see much difference with respect to security. In fact, Nix encourages and flakes formalize pinning of nixpkgs versions. I am sure that there are a bazillion repositories/configurations out there that use a pinned nixpkgs version with known vulnerabilities in glibc, libxml, or whatever.
Besides that, packages in nixpkgs often have known vulnerabilities for months, because of a lack of volunteers to maintain all of nixpkgs.
I love Nix and the large amount of work that goes into nixpkgs. But let's avoid every Nix discussion becoming a 'Nix evangelism strike force' has landed kind of thing. Be honest about Nix' strengths and weaknesses.
Garbage in, garbage out.
It would be nice if Nix could make vulns go away entirely, but you can't keep people from creating buggy packages.
Nixpkg does quite a good job in tracking those issues, imho. https://github.com/NixOS/nixpkgs/issues?q=is%3Aopen+is%3Aiss... is a list of security issues. Most of them generated by automated scans of nixpkgs-unstable.
But as far as I am aware, there's no mailing list or so for receiving notifications upon critical vulnerabilities(?). https://nixos.org/community/teams/security.html mentions github issues, discourse and matrix. Triaging security issues requires significant work and it's a task even more traditional distros like Debian often struggle with.
One thing I'd like to see eventually is an option to nixos-rebuild and other to emit warnings if installed packages are affected by known vulnerabilities. I think that should be doable and would maybe raise awareness and provide most visibility to the issues affecting most users.
https://github.com/flyingcircusio/vulnix does something like this, but it's currently a third-party tool
They need some serious shakeup from that side in my opinion. I've had PRs open for trivial things for >1mth. Issues, security stuff, new packages. They don't even document how to mark something as a security issue. PRs that mention CVE do that, but otherwise I couldn't find anything so for example https://github.com/NixOS/nixpkgs/pull/161198 which includes a security fix is still waiting.
If they can't pull off the reviews with 577 people (https://github.com/orgs/NixOS/people), then they need to start rejecting new ones or relaxing the automerge rules - at the very least on simple version bumps. Flakes at least provide some solution for easily including the 3rd party software without merging.
Only committers can add those labels - it's probably best to flag it in the PR subject line and hope a committer notices it and turns it into a label.
Edit: more...
When it comes to "trivial" changes I think there's an ironic "zone of triviality" where, if other reviewers are anything like me, reviews of more serious bumps will get prioritized. A minor bump is so likely to get superseded very soon that I'll prioritize something that is more likely to cause breakages and require attention. Some package areas (e.g. python packages) do periodic mass bumps of packages to catch the stragglers.
There are of course issues though drawing attention to "trivial" changes that are subtly important like, as you say, security issues.
I would be cautious not to assume the alternatives are perfect https://security-tracker.debian.org/tracker/status/release/s.... It's not uncommon for nixpkgs to get a security fix out before debian.
I think the real issue is that nixpkgs has a lot more "long tail" packages than most package managers, where security issues are grey areas. e.g. upstream aren't great at handling them, making new releases with the fix or god forbid acknowledging the need to be able to backport a fix to a stable branch.
Though I don't disagree that timely reviews would help an awful lot in nixpkgs.
https://discourse.nixos.org/t/nixpkgss-current-development-w...
- so much things are implicit
- inversion of control on steroids
Besides, I love NixOS.
EDIT to elaborate: when you try to read how various things are packaged, it's hard to know where certain things come from (`with` statement isn't helping); you wonder, how on earth can it work when it's mostly empty; names of some functions don't appear in files they're defined.
I can’t imagine taking myself so seriously that I would write a comment like this. This is an opportunity for self reflection more than a lesson worth sharing about Nix.
Yes, Nix is incredibly complicated and needs to improve to see more adoption.
I was just commenting on the weird flex of the OP.
And I love the idea behind it and the programs only available within X directory encapsulation but for me the docs just weren't clear enough when I tried.
I should try again. :-/
I hope for Guix System but so far it's community seems not really more focused than Nix* ones, they are deeply INRIA/HPC centric not much valuing desktops and desktops needs for what I've see and lack of stuff like full LVM support built-in, zfs support does not help much, so so far I judge it not much mature enough for my desire. Guix docs are far better than NixOS ones, however there are no "books alike docs" from zero to sufficiently acculturated in Guix/Nix witch is a big issue IMO.
However they are undoubtedly the future, there is no other sustainable model for the modern world but them. Modern container-mania can't scale down to system level and it's not much better than full-stack virtualization on x86 few years ago, a current fashion with no valuable technical reasons to be, modern packages manager around them (like snap, appimage, flatpack) are just crap needed to push commercial crap in FLOSS world, sorry for being rude, but that's is. Just look at their repo (including things like naming them store) how outdated they are in bundled deps, how many holes punched around they "security by isolation" model to make them usable and still they can't scale. Classic package managers came from another era, are in use, but not much sustainable anymore so...
One thing I still can't figure out is if nix (or flakes) allows you to pin packages to a specific version. Does anyone now if that's possible and how?
E.g. (pseudo code below)
services.nginx.version="1.21.4"
services.nginx.enable=true
What's your thoughts on a NixOS like operating system for FreeBSD? And have you seen any development effort for it (I don't mean the package manager for FreeBSD which already exists).
Frankly, FreeBSD being an entire operating system with userland included seems way more appealing to me than Linux based NixOS - since it seems like reproductability would be even easier/stronger for BSD than Linux.
As far as reproducibility goes, in practice most of the invariants that are not reproducible go down to people embedding way too much metadata into things for no good reason. A BSD base would certainly remove a bunch of variables, but it's not magic reproducibility sauce.
You can supply a particular package version to most services e.g. `services.nginx.package`.
Or if you need to vendor a service, you can copy it into your local repo and use `disabledModules` to prevent the loading of the original.
I haven't used them yet, but I think flakes will make this a lot easier, if my understanding of them is correct.
The main drawback of it is that it currently builds all of your dependencies in one big derivation, so any dependency changes cause a full rebuild. There's some other project I saw fly by which attempts to do a similar thing but split each crate into a separate derivation, but I forgot what it's called and have no experience with it.
[0]: https://github.com/nix-community/naersk
[1]: https://cs.tvl.fyi/depot/-/blob/ops/journaldriver/default.ni...
2. https://discourse.nixos.org/t/cargo2nix-dramatically-simpler...
{
nixpkgs.config.allowUnfree = true;
services.xserver.videoDrivers = [ "nvidia" ];
}
For embedded systems the roads are certainly less traveled. There is decent support for several popular ARM SBCs (https://nixos.wiki/wiki/NixOS_on_ARM), and you can get it onto others with some elbow grease. The official binary cache only provides binaries for aarch64, so you will have to compile everything yourself for 32-bit ARM.
The main issue is that nixpkgs is unapproachable to new users. Yes, there is the documentation at https://nixos.org/manual/nixpkgs/stable/ , and it is quite long. It's not enough.
Every Nix tutorial basically starts off with "this is how you create a new package. You get your source with either ./. or by using a fetcher to download a release from the Internet. Then you use mkDerivation to package the release by writing the install scripts." OK, cool.
Then you want to package a Go program. Cool, let's copy from an example in nixpkgs. Well, stuff in nixpkgs doesn't use mkDerivation, it uses buildGoModule, and it references weird SHA hashes for the vendor? How do you get those? So you go to https://nixos.org/manual/nixpkgs/stable/#ssec-language-go and it doesn't explain to you how to get the vendor hash. It doesn't explain to you why nixpkgs uses buildGoModule and not mkDerivation. So you don't really know how to continue and you feel pretty stuck.
The thing is, a) buildGoModule is a higher-level function that will return the result of mkDerivation in a way that reuses everything that is common to Go modules. So it makes perfect sense from the perspective of a maintainer who wants to reduce the size of nixpkgs as much as possible, which is already one of the largest Git repositories on GitHub.
b) Nixpkgs library functions like buildGoModule, with the need to maintain the separate vendor hash, come from a desire to write pure Nix functions without network access, i.e. there is no need to re-download the vendored dependencies if they already exist in the local Nix store cache. Indeed, the fact that there is only a single vendor hash represents a pragmatic compromise; whenever any dependency changes, the entire vendored dependency tree needs to be re-downloaded and rebuilt; if the hashes of the individual dependencies were tracked, then Nix would understand to only download the dependencies which had changed. The reason why Nix can't compute the dependency hashes of each dependency internally, and only then try to check the Nix store and see if they exist there, is because Nix does not yet support recursive calls: https://github.com/NixOS/nix/issues/13 (nearly ten years old). However, the RFC for adding support was finally merged: https://github.com/NixOS/rfcs/pull/92 and so support will come soon enough.
So basically: Given that most developers need to produce containers to ship to production, and Nix already makes it possible for developers to ship containers, Nix needs to make it easier for developers to build their services/programs in Nix. Program build support is already pretty great for some specific ecosystems (e.g. Haskell) but for most ecosystems (like the Go example above) it ends up being pretty frustrating for one reason or another, due to lacking language features. Once those features are developed, there's a lot of room to make huge improvements in the various lang2nix projects, and then much more of Nix's value will be accessible to developers.