We want to make Nix better
determinate.systems
determinate.systems
1. Approachability for those not indoctrinated in lazy, declarative, functional programming (i.e. Haskell); nope, Nix Pills are not sufficient for the average dev
2. Editor tooling to help guide the writing of expressions (just as anyone learning C# or Rust can use LSP); a better type system for Nix would help here
3. Better documentation for practical things like "Using Nix to manage the dependencies and package a C++ program using Meson", rather than having people piece this together from a bunch of disparate docs
4. A much better CLI UX than `nix-env -qa` and the like (this is ongoing and experimental, but even that broke recently, causing lots of confusion; now it requires `nix --extra-experimental-features nix-command --extra-experimental-features flakes search nixpkgs`)
5. To seriously answer the question: is the Nix language required for the Nix packaging system to exist? Laziness is required, to some degree, but can the next iteration provide an on-ramp which doesn't involve learning a new lang and paradigm? Guix folks sure think so.
I feel like Nix folks have been focused so long on solving the tough problems of declarative, deterministic packaging that they haven't been able to focus on the UX. I also feel like folks for whom Haskell is comfortable may not realize just how absurd it feels to everyone else. Perhaps like the early days of Git.
I do really hope Nix succeeds in this, though; I've been using it, or it's been using me, for several years. [1] and [2] for more info.
This is a pattern I've seen over and over: if a project tries to do something different, the novel aspect gets blamed for issues that would not be surprising in any other context. When faced with a massive awful codebase in Java, well, sometimes legacy code can be a pain, right? It is what it is. People rarely jump to "OOP is a mistake" or "Java is completely unusable". But that's exactly what happens all the time with, say, functional programming.
Pointing to an uncommon choice as the underlying cause of a problem makes for a nice, pat story; the pervasive, diffuse problems inherent to messy legacy code do not. After all, here's a single thing that, presumably we can actually change! Nix being functional is an objective fact while people can—reasonably or not—disagree on how well-designed and well-written Nixpkgs-the-codebase is. Besides, even if we could agree, what could we do about it? We all know how much momentum legacy code has and how hard it is to make anything beyond local improvements. Being a socially convenient story doesn't mean it's a particularly strong explanation, but it does mean that people will believe it even if it isn't.
Nix pills, the official manual, etc. aren't really there yet--they talk about all kinds of paradigms that might (?) be going away like nix-env and its ilk. It's just extremely painful to track down the current best practice and way to do things in nix--you're bouncing off all kinds of docs and finding multiple conflicting or out of date ways to do things. Someone needs to give folks a slim book about nix flakes and the flakes way of doing things, the nix language, and a tour of nixpkgs, and IMHO there will be a lot more people using it.
If it was bad why show it to me at all?
So to most programmers, imperative code is easy and FP is hard, but on an objective basis both can be simple. In a way FP may be simpler (according to the talk), as e.g. mutable variables complect/mix together values and time.
The biggest barrier, as I see it, is not actually Nix or Nixpkgs code, but the way NixOS differs from every other Linux distro. Most of my delving into Nixpkgs has been motivated by getting something working that would be trivial in a traditional distro. I agree the Nix language is a red herring, what's really needed is to make work other than "adding a package to Nixpkgs" easier. Actually usable introspection tools that the GP mentioned would be great.
The same thing we do for other paradigms (system architecture, programming concepts, etc.) - find and document common patterns, then find ways to transition to the patterns away from things out of pattern, and lastly introduce automatic refactorings that do fix these things in bulk.
I definitely agree with you on the evolution thing. Homebrew is a similar DSL for installing packages on macOS. It has the problem where things get deprecated from the DSLs (Casks and Formulae) which leads to not being able to run various commands until previously installed software is fixed.
I'd just like to take a moment to reiterate that object-oriented programming is a mistake produced by cargo-culting half a dozen different styles used by competent programmers (including the devolpers of smalltalk and its relatives, from which OOP appropiated its name), and that Java is completely unusable by anyone with a sense of self-respect and a awareness that programming languages can be better than least-common-denominator noe-COBOLs. I'm not familiar with the specific codebase you're refering, but it would probably be at least less massive and less awful if it had been written in a more decent language, like Haskell or C.
Java is a tiny language with not too many concepts going around sitting on top one of the best runtimes with one of the biggest ecosystems. It is a very sane and good choice for almost every program out there, especially that empirically we have absolutely zero evidence for any significant difference between language productivities, other than managed languages being a boost (so your mention of C, which is arguably a language with so many faults one could fill a whole book about, is frankly, quite laughable).
I'd love to hear from someone deeply familiar with Nix and Guix about laziness.
I'm deeply familiar with Nix and I've concluded that lazy semantics is absolutely critical for a configuration language. It lets me refer to other attributes of my configuration from anywhere. For example, I can refer to port number from my whatever service in my firewall. Nix's system of overlays depends on laziness too to provide efficient late-binding familiar from OOP.
I don't need to topologically sort the evaluation of the various inter-dependencies of my configuration. So long as there exists an evaluation order, laziness finds it.
Laziness is compelling enough that I managed to convince the author of Jsonnet <https://jsonnet.org/> of it when he was designing it, and in turn he helped me design what is now known as overlays in Nix.
I don't even understand how Guix manages to work without laziness, though clearly it does somehow. I'm curious as to how that is possible, though I fear I will only ever truly understand by diving into Guix.
As an OOPer, inheritance is one of the things that is generally trotted out as the worst aspect of OO languages. This leads to the mantra "favor composition over inheritance". Much has been written about why and how. But it seems to cause the same pain points that people talk about with respect to laziness in Nix. "Spooky action at a distance" is one of my favorite ways of describing some of these problems.
(I'm not super familiar with Nix, so I could be misreading the points about laziness. Please forgive me if so.)
I certainly think people should exercise caution as this could probably be abused quite badly. Maybe people will come up with better designs in the future. However I will say, that the method it has replaced was much worse. The old way was adhoc and almost always failed to achieve ones desires in all but the simplest cases.
Don't just take my word for it. Peter Simons also agrees: <https://github.com/NixOS/nixpkgs/commit/3c8b33eee442fd573d47...>
That’s not a property of laziness but of how Nix evaluates configurations. I think people overvalue what laziness brings them because they attribute to it things which could be done equally well with an eager language.
Being able to write
let usercfg = { username = "nix-user";
homedir = "/home/" + usercfg.username;
};
in usercfg
Strict languages can simulate these non-strict semantics by using functions of no parameters to delay evaluation, but you lose the efficiency that comes from the memoization of record fields that you get with lazy evaluation.
P.S. The above would perhaps be more likely be written as
rec { username = "nix-user"; homedir = "/home/" + username; }
There's a case to be made about improving the state of the Nixpkgs APIs though. Stabilizing the APIs and documenting it more thoroughly would dramatically change how people perceive the complexity of Nix. Experienced Nix users grep the Nixpkgs repo to learn about its APIs, but that's far from ideal for newcomers who won't even know where they should look for information.
Still, despite its reputation, Nix lowers the bar for software packaging. Package definitions tend to be more concise, customizable, and reusable than the competition. Nixpkgs contains many useful utilities for packaging and there's an abundance of real world examples contained in a single repo. I also frequently work with RPM, but Nix is much easier in comparison.
When documented examples began steering me towards the still experimental nix flakes and using the sorts of with ever more flags required commands, I bailed and built it using salt. I want to like nix but the current state of things was a massive put off.
There's a push to make them not be experimental anymore since so much of the community including myself is using them.
If I’m learning something as big as nix with all the facets it has, I don’t also have time to learn it’s entire community history. It’s not a good thing that I need to do that in order to make a decision about how much to trust experimental features of a tool when I’ve not even been using it more than a few hours from the moment of downloading the iso and installing it. It’s a put off to someone new coming to use nix for the first time.
Yup. I can't even imagine what could I accomplish if I'd have something like a "python library" for nixpkgs. I still use nixos but gave up fiddling with nix for a while, after trying to wrap some stuff I am using/developing.*
* (Yes, I know it's easy. I just need flakes or correct amount of "lib: prev: { self"s. Sadly not for me.)
The majority of communication I've seen from Nix maintainers over the last few months has been about exactly this.
The submitted post is about exactly that while Nix powerful, they want more people to be able to use it without having to pay a steep penalty to learn it.
Nah. The logical conclusion of this is that Nix should be some Java-like, Python-like, or whatever else qualifies as something that the average developer knows.
Nix needs to be declarative which rules out the average PL paradigm.
> […] , but can the next iteration provide an on-ramp which doesn't involve learning a new lang and paradigm? Guix folks sure think so.
Does the average developer know Scheme? I think you just played yourself.
I think most of the docs problems could be solved with the meta-strategy of “deeply understand all the things the Rust ecosystem got right with docs, discovery, and general pedagogical structure, and replicate that”. Actually have a canonical set of docs. Link liberally between packafes and stdlib. Figure out typing so you can get those hints and links in your IDE. Fix the package docs so they actually tell you what functions the packages export, instead of relying on the wiki for instructions. Etc.
Basically, pay a couple excellent docs people for a few years.
I found the language a bit weird but not terrible to learn. But if it’s possible to build some porcelain for common usecases that skips there need to grok the language, that could be quite transformative. I do wonder if the “dev env/packaging” use-case might be simpler than the “run and configure a whole OS” usecase.
I think every project would love having the money to hire someone full time to work on the documentation and evangelism.
Learning to use a lazy functional programming language remains relatively easy - we use to teach the basis in Ocaml in three hours when I was a student - and Git shows people can tolerate awful CLI tool UX.
I think your point 3 is where most of people issues with Nix lie except I wouldn’t limit myself at a lack of practical enough documentation. Nix documentation pretty much just doesn’t exist period.
If you want to use Nix, you will have to go through the packages or the implementation to understand what’s happening. That’s not very welcoming especially when you consider that’s the other Linux distributions which tried to do things differently in the past and could be a source of users for Nix OS, first of them Gentoo, have absolutely stellar documentation.
It's the easiest programming language in the world.
(Which is a problem when people start building huge projects like nixpkgs with it. It's not a language fit for programming-in-the-large, just like Javascript isn't.)
nix flake show templates
I've only had to add a few lines to my configuration over 10 PCs and laptops.
I have trouble understanding your question. Why do you want to use Nix if you use a system which comes pre-configured to do something else?
We're working on making Nix more accessible and producing good and usable, production-ready workflows so teams can just pick it up and go. I'd love to hear what y'all think, to help make sure we're going in the right direction.
Look at this:
https://nixos.wiki/wiki/PostgreSQL
Now, the real "workflow" with nix is look at other setups, or just look at the code:
https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/s...
And look at the code is the only reliable way to see what exactly is supported.
My second gripe is the fact is hard to see what nix do. Today I hit this trouble:
https://www.reddit.com/r/NixOS/comments/x46w98/why_new_user_...
The thing is: Nixos not tell what is doing. `verbose` is too much noise.
What I wish now is something like:
nix change nothing (if my changes somehow don't do anything, like in my issue)
nix setup users...
nix setup postgresqlI do think command line options to do what you ask would be nice, but in order to keep the declarative reprudicibility you get with nix, those commands would need to operate on nix config rather than the system itself. They could update the config then update the system based on the config, so it could look like one step even though it is two behind the scenes.
> nix change nothing (if my changes somehow don't do anything, like in my issue)
I miss this feedback greatly having used Ansible since 2013 and only this year diving into a single NixOS installation as my intro.
Does Eelco's departure from Tweag have any impact on Nickel as a possible replacement for nix?
There are lots of fragmented attempts at making nix easier/more immediately valuable/lower barrier to entry. Some of these are personal projects, some are businesses. Some of these just make nix easier, and some attempt to put something in between nix and the user. I'm thinking about things like Cachix, Flox, and divnix (formerly devos). How do you see your work interacting with these?
One of my barriers to adoption at work has been ensuring maintenance continuity. That's always going to be the case for tools that aren't ubiquitous, but I worry with nix that teams without my assistance will revert to other tools they know better, even if I've invested a good bit of time in teaching them enough to keep things going. Do you seek to address that?
I am working on a very large project where multiple solutions have been invented to solve problem which are all handled by Nix:
- Bake in the repository the clone of the dependencies.
- Compile the tool chain to create reproducible builds.
- Create "artifact" builds, to avoid large recompilation times for many developers.
- Use various tools to cache and distribute builds.
- Use a bleeding-edge version of some dependencies.
- Use Node, Java (Android), C++, Rust, …
If only Nix were to run natively on Windows, this would check all the boxes.
This is also due to the fact that I tend to value Graham's posts highly :)
However, one of the bigger problems here is the general API design of Nixpkgs. Most of the function interfaces use some named parameters and then use `...` to accept any additional parameters. All of those then get passed down to some other function, and on and on. This makes a mess! It is hard to tell what functions use what parameters, and none of them can restrict their interface.
We could see some significant DX here by creating smaller, more specific interfaces that actually restrict their inputs.
Nix expressions are almost always[2] short-lived snippets that evaluate to a data structure, not long-running programs where the distinction between "static analysis time" and runtime is extremely relevant.
In fact, deriving the full set of potentially relevant type constraints is only possible at runtime due to how the import system works and the shape of most expressions. If you pick a single file from nixpkgs without context, you can't get much information from it at all (other than a lot of unknown types being passed to things that accept an unknown type) - it's only in the context of an evaluation of a graph node that uses that file that you can infer anything meaningful about it (and even then that information is only relevant in that context).
I think what people actually want is a way to derive more sensible documentation information from the code so that the various builders etc. could have checked documentation pages, as well as tooling for auto-completing members of attribute sets, names of (formals) function arguments and so on.
fwiw, there's an alternative implementation of the Nix language[0] that we (TVL[1]) are slowly open-sourcing at the moment, and we're aiming to design it in such a way that things like an LSP can be implemented on top of it, and to be able to dump out static information from a context that can help with documentation etc.
[0]: https://cs.tvl.fyi/depot/-/tree/tvix/eval
[1]: https://tvl.fyi/
[2]: Yes, I'm aware that Nix is turing-complete, but surely nobody would write a web framework[3] and HTML templater[4] in it.
[3]: https://cs.tvl.fyi/depot/-/blob/web/bubblegum/README.md
This is exactly where types would be the most beneficial, and what makes nix so hard to read. If a nix function could specify the types of its arguments, then I might have a chance of better understanding it.
Just because most nix functions currently don’t have many constraints on their inputs doesn’t mean that they couldn’t in the future.
> it's only in the context of an evaluation of a graph node that uses that file that you can infer anything meaningful about it (and even then that information is only relevant in that context)
That’s how nixpkgs works now. But we can make functions that work only in more limited contexts.
That's a great start, but it would be nice to know what the expected type of the member is! Often it's another attribute set, a list, or even a function, and it's impossible to tell what type is expected without looking at where the member is used.
However, this has massive runtime cost (especially in hot code paths), so some sort of spec-like annotation system would be cool. We'll see how it shakes out over time ...
As for error messages, we might already be able to do big improvements in Tvix. Our bytecode has pretty exact tracking of the source spans that things come from all the way throughout, so we could trace things like a type error being raised somewhere to the last time that value was passed through as a function argument from user code etc.
This error message business will be a whole area of development of its own however, for now we're just making sure that as much of the relevant information is available as cheaply as possible.
What is the reason for all this machinery? I went with the recommended multi-user install, should I have just used the single user mode instead?
> new Unix group
This is used for the daemon, so it doesn't run as root and expose your system to Nix build code.
> adds a separate APFS volume
I think this is required because of macOS security restrictions preventing direct modification of the root directory. The Nix store has to be housed in /nix because all references to runtime dependencies in the store are absolute paths in /nix, and that can't really change per system because it would break caching and reproducibility. The separate volume is added to /etc/synthetic.conf so it can live in /nix.
> installs a daemon
For multi-user installs, this allows non-privileged users to add build outputs to the Nix store, which ultimately allows these users to share build outputs. More info here: https://nixos.org/manual/nix/stable/installation/multi-user....
This is true, but in some sense Nix on Apple Silicon was a missed opportunity. It started as a blank slate (fresh binary cache) and it would've been a good opportunity to move the store to a writable path like /opt/nix. This would have solved the whole dance needed with volumes and synthetic.conf. I know that there are infrastructure issues (like Hydra using a single Nix store), but it would've made the macOS Nix story so much better.
It's a shame that only Apple can make firmlinks, because that would've been another possible solution (/nix could be a firmlink to the actual store location).
The same problem occurs on e.g. Fedora SilverBlue, because you can also not make arbitrary root directories. But at least on Linux, you don't want to throw away more than one decade of a x86_64 binary cache.
That breaks caching cross-compiled outputs and x86_64 outputs, which do still run on Apple Silicon.
This is the main reason why I don’t use Nix on the Mac anymore, I just don’t want the pile of hacks that is necessary on my system. And I am a former nixpkgs contributor. Many people will just shrug and install Homebrew.
Although it is my hope that rfc 17 eventually makes it through: https://github.com/wmertens/rfcs/blob/master/rfcs/0017-inten...
Sure. But it's still nice that e.g. on Linux x86_64 you can use a nixpkgs commit from 2015 and get all stuff from the binary cache.
I think the multi-user install is a better Nix experience, even if the install process is spookier.
Like is mentioned elsewhere, Docker does similar contortions to install itself. I wonder if it would have been more palatable if the Nix installer was less forward about what it is doing?
I bet most wouldn't consider it spooky if the installer didn't print it out.
If docker started printing it, I bet lots of people would complain it seems similarly complex.
Nix wants to use /nix, but Apple wants macOS to be locked down and secure.
The compromise was putting /nix on a separate volume.
> I discovered it creates a new Unix group, ... installs a daemon
This is at about the same level as Docker, fwiw.
Why does a separate directory need a whole other volume? These are just files, right? Why do they need to be on a separate volume?
Nix achieves this by storing packages in a path as some hash of its inputs. -- This then allows either compiling or downloading a package, with confidence that it will behave the same way regardless.
But, since the files were under /nix, if you put them under /opt, then you wouldn't be able to make use of the compilation caches for /nix.
Until a very short time ago there was no binary cache for Apple Silicon Macs.
I don’t really understand the complain in this thread. Using an APFS volume just works and is entirely transparent to the user.
People could just learn instead of complaining they don’t understand.
/nix could still be used if it's on a separate volume. Which still seems less nice than just having /nix on the same volume.
You can install Homebrew other places, too, for that matter.
I'd only tried NixOS (bounced off, couldn't get X-Window working even following tutorials to the letter) not Nix on macOS. "Must be installed in a specific, root directory" is a hard no for me, when it comes to add-on package managers. That's one hell of an odor.
> That's one hell of an odor.
That’s very much fine. They do that to be able to share the cache between system. There is nothing magical about the root folder by the way. It’s just Apple being annoying.
Nix could switch to an alternate location on macOS (e.g. /opt/nix) but that has a lot of downsides for interoperability with other systems.
Using /nix and a separate group and daemon means the store can be read-only and be protected from modification in several ways. This is pretty helpful, as a lot of tools try very hard to write "next to" where they are installed -- corrupting the Nix store.
I sort of wonder if it would be more palatable if the Nix installer was a bit less in your face about what's going on? This would be similar to how Docker's works.
Nix is a package manager, yes, but it's more than that, it's a generalized build system.
> So you need non-admin users to be able to use it too
The build daemon and the user are used for privilege separation. The separation goes both ways. Users can't write directly to /nix/store and Nix can't write outside of /nix/store during build.
If anything, it's there to make things less invasive. It's nothing like the Docker daemon, which is a proxy for root.
Additionally, the daemon doesn't do anything unless users request that a package be built.
Yikes? Well then... how would you solve remote binary caching with something like Nix on a platform such as OSX without userns remapping support?
> I discovered it creates a new Unix group, adds a separate APFS volume, installs a daemon. This was too invasive for a tool I was unsure if I even wanted to use, so I uninstalled it.
Since you'd already installed it, wouldn't trying it in some capacity before uninstalling it have made sense?
> What is the reason for all this machinery
Enforcing reproducibility basically.
I don't know all but...
The daemon and APFS volume otherwise readonly /nix that only the daemon can write to can't be created.
The group is probably for the daemon to be able to write to /nix.
The path /nix is important because the remote binary cache paths will miss otherwise and you'll compile everything from source.
> I went with the recommended multi-user install, should I have just used the single user mode instead?
I'm guessing it would work in the way you want, but I always opt for the daemon.
Fair observation. I installed nix as a prerequisite for DevBox, discussed here https://news.ycombinator.com/item?id=32600821 . I thought DevBox sounded really cool (and still do!), but the Quickstart took frustratingly long, and ended up not working. Faced with the prospect of debugging it, I opted to cut my losses and uninstall it instead.
That said I'm very much open to trying nix again in the future. Also I want to acknowledge how much effort went into getting /nix to work on the Mac; it appears that was a heavy lift indeed.
Despite the reputation, I feel that NixOS or some derivative of it has the power to become the best distribution for non-technical users in the long run. What NixOS has done is effectively built an interface to every component of a modern Linux system, all that needs to be built is a user application to take advantage of it. Of course, there still needs be some improvements in Nix itself for it to blossom into its final form, but I really see a path to greatness here.
I have often thought about creating a simple unified Win2K-esq or BeOS-like X11 WM/DE specifically for NixOS but unfortunately I lack the time/motivation.
It wouldn't be able to handle arbitrary manual changes to config at the beginning, so either there would always be a protected generated config region of the whole config and the distro would remain relatively standardized, or an alternative evolution would be a low-code user friendly UI that could map to any possible nix config.
It is also complex enough to require a modern Language Server.
We adopted it in an organization of ~100 engineers, and the only way it’s been possible is having a full-time Nix team writing custom Nix functions specific for our environment and projects. That team also does “Nix help desk” work for one-off questions.
Once it works, it does a great job of hermetic builds, easy Docker images, easy to add cross-repo dependencies mixing C++/Python etc. But there are too many rough edges I can’t recommend it in the general case over Dockerfiles, Bazel or language-specific tooling, Cmake, etc. Pick something simpler, ideally whatever is popular for your language.
Knowing functional programming helps but Nix still is a hard language to learn. I have a lot of Haskell experience but the Nix language was very confusing for me. At first I couldn't even tell a variable name from a keyword (it is a convention to call variables "self" and "super" in some contexts). The Nix language has been created for research in build systems and has evolved over a long time, so it has accumulated quite a lot of idiosyncrasies and cruft.
I hope that a simpler typed language like https://github.com/tweag/nickel replaces it.
I've had concerns about Nix's governance in the past but it sounds like they may be going in the right direction, so I'm excited to see what they're planning.
You can use bob[1] if you want a build tool which uses Nix to install dependencies in an easy manner: just list the package names for a task and then they will be installed.
I'm looking forward for all the changes in Nix ecosystem and it's a good sign the fact that they also started working on an initiative to improve Nix documentation which was spread all over the places.
Suggestion: make Bob installable with Nix!
Already created an issue :)
But bob intends to be more than a dependency manager, it wants to be fully-fledged build system similar to Dagger, Bazel, Earthly etc with features like:
* build pipelines[1]. You can define and manage your build pipelines in a familiar YAML format. You also get caching for your targets and so on.
* isolated environment. We completely clear the environment for a task. I see with devbox that if I do a `printenv` or an `echo PATH` while I'm still in an active devbox shell, I see variables from my local system which can affect reproducibility.
* workspaces: manage multi-repo setups in an easy manner
> Many projects have build instructions that depend on downloading stuff in a complex manner that cannot be easily abstracted/shaped to what Nix expects.
Are you referring here to packages which are not available in Nixpkgs?
A bespoke YAML format is arguably not simpler than just "here's the equivalent Nix expression". But, sure, can be more accessible for some people.
But, providing a nice facade over the Nix language doesn't help with cases of where Nix can be difficult to work with.
This is true. If you reach the point where you need to write your own Nix expression you currently can become stuck.
But I think these are problems that are waiting to be solved. There are folks are already working on a new language tutorial at nix.dev for example and other tools that can abstract some of the problems we have now can be built. I believe we are still in an early stage when it come to reproducibility and we just have to keep working at it.
The basic way to fix this with nix is:
1) download the thing with a fixed ouput derivation: a derivation who's source has the URL and checksum of the artifact
2) copy the artifact from the nix store where said build tool stores it
But you can also add .nix expression files in the same bob.yaml that can be used to build your packages when a package is not found on Nixpkgs.
The only hard requirements I can think of are algebraic data types with exhaustive pattern matching to go with, row polymorphism, purity and good inline documentation support.
I don't know if a good enough hostable language exists or if it should be a new version of Nixlang, but almost every single annoying problem that makes me go "Nix is getting in my way" can be traced back to the lack of a good, powerful type system leading to a house of cards situation whether it comes from nixpkgs or entirely of my own making.
Perhaps something like heroku buildpaks (https://github.com/railwayapp/nixpacks ?) would help devs get on the Nix train.
I believe live demo and CI/CD pipeline might be some of the best use cases to get people interested in Determinate Systems. Don’t get distracted by language features and syntactic sugars. If it works, people will learn.
I'm considering Nix but don't like the custom language. I wish Guix were. more popular and less extreme on the freedom side. Chances are I'll just do a fresh Arch install or maybe try Debian rolling.
These days I like to install almost all software and libraries by hand in my home directory. All I really want is a solid system that basically boots fresh then loads my home directory. Maybe I should do a net boot of some sort?
Any other suggestions?
you can also manage tools with asdf if it makes it easier.
I personally run Fedora + Flatpaks + asdf and it covers everything, and is stable, up to date, and has a new release every 6 months. it feels midway between a rolling release and. apoint release
The "external dependencies" thing shows how Nix can be a tough sell to those who don't click with it.
I'd say that the other solutions are:
- a README which has a bunch of "apt-get install" commands you can copy-paste from,
- a "setup.sh" script that installs things,
- a container or VM image with the toolchain setup,
- or a tool like asdf.
IMO, Nix is a much nicer solution than these other options, and has the benefits, avoids the downsides of these (but an entirely different set of downsides).
I think they both also have downsides where, if your software does things in ways they don't like, it's difficult to use them. -- And, your team is going to have a tough time using the tool without sufficient expertise.
As I understand it, Bazel's emphasis is more about compilation, where the big win is building projects where you've got a team big enough (& a project complex enough) that dealing with Bazel saves more time than it costs.
Whereas with Nix.. I think one good use case is "declare & lock the dependencies for this toy project", which then allows picking the project up without having to update your code following updates to the system package manager.
It's not because of the nix language, It's not because of the CLI, it's because everything is scattered, you have to consult many places to find out how to do things with Nix, here is an example:
Usually, when I need a new complex program, like Steam, I first check the system-wide configuration [1], the wiki [2] and the package list [3], if I just want it on my user, I need to check if Home Manager has an option [4], if it doesn't, I can try using the "home.packages" option. Now, if I need to override something on the package, I need to remember how to do it with [5] [6] (while checking the source code for the package in parallel to find the options).
And then sometimes, on very rare occasions, I need to fine tune something with the nix language, so I need to check the builtins/lib docs [7], but some builtins are not there, so I need to either use nix-doc [8] or find the docs inside the code-bases [9] [10] (they are split between both repos)
For me, this is one of the main pain points of using Nix / NixOS that needs to be solved.
[0] - https://github.com/shiryel/nixos-dotfiles
[1] - https://search.nixos.org/options
[2] - https://nixos.wiki/wiki/Steam
[3] - https://search.nixos.org/packages
[4] - https://mipmip.github.io/home-manager-option-search/
[5] - https://nixos.org/manual/nixos/stable/#sec-customising-packa...
[6] - https://nixos.org/guides/nix-pills/nixpkgs-overriding-packag...
[7] - https://teu5us.github.io/nix-lib.html
[8] - https://github.com/lf-/nix-doc
For example, esbuild is written in Go, which is compiled to a different binary for each system. The NPM for esbuild has 21 optional dependencies, one for each binary that it makes available. A post-install script [1] chooses which dependency to install.
It seems like a lot of work for the maintainer? But most users don't need to care.
It probably helps that the Go SDK builds static binaries.
[1] https://github.com/evanw/esbuild/blob/master/lib/npm/node-in...
This complicates working on node projects with nix in my experience.
Is there a way to tell npm "no, just use my binary"?
Best i've found is letting npm download it and then overwriting the npm binary with the nix one in a postinstall script.
Why you ask? Npm doesn't always have an aarch64 or arm32 binary.
Nix being a source based package manager typically does support it or at least makes it possible by overriding build/configure options.
Also, I wonder if nix would be a suitable tool for generating npm packages that contain binaries? Maybe this is a way for people to benefit from nix without actually using it.
The "I have a binary already, just use this one" is in response to the problems NixOS users face dealing with helpful software like this.
NixOS doesn't provide shared libraries in the standard locations (for good reasons). Binaries that tools like NPM download will link against shared libraries in the standard location.
It's not really a question of "providing the binaries would help", because it's about assumptions of where the shared libraries are.
1. Does every nix-shell require building the whole world from ground-up? Seems impractical to me. 2. What is the right approach?
This is not to bash Nixpkgs, because I installed NixOS and it took me 10min to install a whole OS with Sway, Neovim and some other tools. I guess I'm doing something wrong. But, on MacOS, nixpkgs was not a pleasant experience at all.
It's the same as homebrew:
If there's a cached binary already, it will download the cached binary. If not, it will fetch the source and build it.
The QMK nix shell needs different GCCs for different targets. For the QMK shell.nix specifically, you can disable the targets you don't need (e.g. if you don't use a teensy board).
I have to learn more about the nix language, but looking at the shell.nix seems that I have comment/remove lines from 25 to 30 according to my board.
{ avr ? true, arm ? true, teensy ? true }:
Exactly that? Lol :)
https://nixos.org/manual/nix/stable/command-ref/nix-shell.ht...
I still will try when I get to my macbook. Thank you!
I think the idea that Nix is difficult to get started with, and undocumented, isn't a surprise to the community. -- These were some of the most repeated points in the community survey. And the topic of the submitted post.
That said, I do think Nix has seen a lot of growth over the last year, and I see fewer "nix is so easy" comments.
You can use nix-build on a shell.nix though i'm pretty sure.
Agreed! Most repos with a shell.nix and heavy dependencies use their own binary cache with the service cachix.org (free for public repos).
> But, on MacOS, nixpkgs was not a pleasant experience at all.
Were there reasons besides the QMK nix shell building the world?
Created an issue:
MacOS has issues with binary cache because of apple's requirement on where macOS can be run.
pkgs.yourpackage.overrideAttrs { prev: {
src = fetchurl {
url = "...";
sha256 = "...";
};
}}
With flakes you can just add a 'ref' or 'rev' to the URL:
nix run "github:juser/software?ref=v0.4.1"
NixOS is a bootable OS build on top of Nix.
Nixpkgs is the default package collection used by both Nix and NixOS.
If they do manage to create a system where Nix is hidden, and end users never have to directly touch it, I think this could work and make my work never see the light of day.
But I have my doubts that they will be able to do that, and it boils down to one simple reason: declarative is not powerful enough.
Don't get me wrong, for 90%, possibly more, of use cases, it's enough. And it is preferable to keep things declarative as long as possible, so much so that my build system will have a way of restricting itself to purely declarative code when possible (and will error otherwise).
However, when more power is necessary, it is required; it is not possible, by the definition of Turing-completeness and declarative, to do the same thing with a declarative language that can be done with a Turing-complete language. So when that power is missing, the only option is to work around it.
This is why Nix is really just a front for bash. For all of its faults, at least bash is Turing-complete.
But this fact means that there will be things that Nix will fail to do on its own. I suspect this means that the abstraction around Nix will always be leaky. Maybe it won't be unacceptably leaky, but I'm not very hopeful.
I think Nix would be better served by doing the following:
1. Rewriting the language. This would require an auto-transformer to the new language in order to not throw away the entirety of nixpkgs, but transforming a declarative language to a Turing-complete one is easy.
2. Spend gobs of time on user experience. Make the usual commands short, easy to remember, and easy to use with few options. Make usual things easy and powerful things safe. For example, to make a powerful thing safe, make sure that screwing it up will not screw up their system.
3. But not only should they be safe, they should feel safe, giving users every opportunity to back out without consequences. This is where git goes wrong: it does not feel safe, so users are scared of it. [0] I believe Nix is the same way, even though it is safer than git.
4. Spend gobs of time on documentation. Use Fred Rogers' list of rules for talking with children [1] in each piece of documentation. This will make it easier to avoid the trap of not explaining something the user needs because the writer forgot that they didn't know it. In essence, the documentation should treat new users as Fred Rogers treated children: ignorant, but capable.
5. The documentation also needs to have a different focus. Instead of focusing on how great Nix is, it should focus on helping users get stuff done. Nix enthusiasts should be able to say, "Oh, you want to set up a development environment for your project? Great, go to such-and-such tutorial. It will tell you exactly how to do that even if you don't have Nix installed." This should be done for as many use cases as people have, including less common ones. Some examples: using Nix as a build system, using Nix to install multiple versions of glibc and how to switch between them, using Nix to set up systemd, using Nix to replace a Docker container in production, using Nix to distribute builds, etc.
[1]: https://www.mentalfloss.com/article/547536/mr-rogers-rules-f...
I fall heavily into the use less powerful languages and "liberties constrain, constraints liberate" camps.
I'd be very interested in a real world example of this, though I know that class of examples is hard to come by.
For what it's worth, so do I. That's why my build system will allow users to set those constraints. It's a great idea.
> I'd be very interested in a real world example of this, though I know that class of examples is hard to come by.
There are many, but I'll say one: anything that requires iteration until a fixed point is reached. A concrete example of this is building documentation in LaTeX.
Or it may even be simpler than that. Maybe your build requires some sort of special code should two arguments have specific values. In other words, edge cases often require Turing-compete power to resolve.
I was going off of the article and Nix documentation that says Nix is declarative. But if it actually is not, then there shouldn't be a problem with power.
However, Nix proponents shouldn't claim it is declarative either.
When I think of declarative, I think of a language that has less power than a Turing-complete language. Think Make without bash or any shell. Or HTML without CSS.
In other words, the Nix community seems to use "declarative" as meaning "without side effects" where some programmers, like myself, hear "declarative" and think of something like HTML without CSS or JSON.
Nix is Turing-complete. This was even sometimes considered a problem because it makes the language too powerful:
What makes it declarative is the lack of side effects and laziness.
Even NixOS itself doesn't do all that much actual programming in Nix, most of the code that does stuff is written in C++. The Nix language itself doesn't let you escape out of its sandbox.
This confusion is not good for Nix.