I've got one where "deploying" means updating a few version strings and image reverences in a different repo. The "build" clones that repo and makes the changes in the necessary spots and makes a commit. Yes, the side effect I want is that the commit gets pushed--which requires my ssh key which is not a build input--but I sort of prefer doing that bit by hand.
Instead of debugging code, the team would have to spend significant time maintaining the build system for the build systems sake. Don't get me wrong, I want something nix-like in my toolbox. I want to love nix. But I wouldn't dare to argue my team to commit to the world of pain that comes with it.
There's a good reason that nix didn't see wide adoption in the industry.
read and setup store/gc settings work for you. do not use nixenv nor nix profile.
I wonder if you somehow ended up eval'ing many versions of nixpkgs?
your nix store weighs at 100 GB
¯\_(ツ)_/¯ outside very constrained devices, who cares? I just checked my NixOS dev VM that I have used for months now and cannot remember when I last garbage collected. It's 188GiB, but I have many different versions of CUDA, Torch, etc. (the project I'm currently working on entails building some kernels for many different build configurations), and I run nixos-unstable, where a lot of stuff changes, so generations are pretty unique.
A 2TB NVMe SSD is just over 100 Euro. Caring about 100GiB seems to be optimizing for the wrong things.
I completely agree on embedded machines though. Just deploy it by copying the system closure, garbage collecting anything but the previous closure for backup, it'll be pretty much the same size as any other Linux system.
I don't know what kind of package manager you were using, but I've never seen an update take a good part of an hour before Nix.
> outside very constrained devices, who cares?
Seriously, are we going to shame people who can't afford to buy lots of storage?? My smaller laptop has only 250GB, but that's freaking plenty if I stick with apt. But I can barely run Nix on it.
It's not just storage, though - storage may be cheap but once your machine is at capacity (the physical space in laptops is an important constraint) you have to replace perfectly good hardware to accommodate absurdly space-hungry software (looking at you, Vivado).
Also, don't forget that not everyone has always-available, fast, reliable, cost-free internet. By rural standards my connection's very good, but 100gb would still tie it up for several hours, assuming I didn't need it for anything else in that time.
Digital wastefulness is a problem, and I do think we need to take it more seriously.
Except that Nix does not download 100 GiB under unless you are installing a gazillion packages. First, Nix downloads compressed output paths. Second, it's not like Nix packages are substantially larger than Debian, Ubuntu, or Fedora packages. The extra storage space comes from (1) Nix keeping multiple generations to allow you to roll back to previous versions of the system -- if you break something, you can always roll back; (2) people using multiple different versions of nixpkgs, which could lead to having multiple versions of system libraries.
(1) is a feature of Nix/NixOS, if you want to use less space, you can trade off the ability to roll back for space. You could always garbage collect everything except the current generation and it would be similar to other distributions. For (2), avoid using multiple nixpkgs versions.
I generally like keeping around a lot of generations, etc. so I don't mind my history of NixOS systems keeping 100-200 GiB. But if you care about space, garbage collect and it won't take up that amount of space.
Pretty much all popular package managers. APT/dpkg, DNF/rpm, pacman, etc.
I have just updated one machine to the latest unstable. It updated 333 packages, a substantial part of that system. It took 1 minute and 50 seconds, most of it downloading. So, not sure how it takes a good part of an hour for you.
Seriously, are we going to shame people who can't afford to buy lots of storage??
I'm not shaming anyone. Just saying that 1 or 2 TB is pretty normal nowadays (outside Mac, because Apple makes you pay for it). At any rate, you can make the size pretty similar to any other distribution. It's not like glibc or GNOME takes up substantially more disk space on Nix.
If you end up using 100 GiB of storage, you are either keeping a lot of system generations around or you somehow have different nixpkgs versions in your system's closure, ending up with duplicate versions of glibc, etc. If the former is the case, set up automatic garbage collection and the space use will be far less. E.g. on one machine I have only three NixOS unstable generations and the system is 18 GiB (which includes a bunch of machine learning models, etc.). It would probably be substantially less on NixOS stable, since there are less differences between generations (e.g. I have qemu, webkitgtk, etc. three times).
Total size of installation is roughly comparable between NixOS and, say, Ubuntu.
My laptop's Nix closure of 1 generation is 33 GB. My desktop Ubuntu has 27 GB (20 GB /usr + 7 GB in /var, where snaps and flatpaks are stored).
Indeed the disk usage of Nix comes from multiple generations. Every time there is a new version of glibc, gcc, or anything that "the world" depends on, it's another 33 GB download. Storing the old generation is entirely optional. The maximum disk space needed is 2 generations.
Updating Ubuntu to a new LTS version almost always costs me multiple hours, caused by interleaved questions on how to merge changed config files in /etc (which unfortunately one cannot seem to batch), apt installation being rather slow, and during recent years, the update generally breaking in some way that requires a major investigation (e.g. the updater itself dies, or afterwards I have no graphics). On NixOS, these problems do not exist, and the time to update is usually < 30 minutes.
We use Nix with Cachix in the team I currently work in. We use a lot of ML packages/kernels, which are nearly impossible to manage in Python venvs (long build times because we have to patch some dependencies, version incompatibilities, etc.). Now you can set up a development environment in seconds. The nicest thing is when we switch between branches we automatically have the state of the world needed for that branch (direnv yay).
It was some work to set up, but it saves so much time now.
Right now I wrote a bash script to check for nix, direnv, git, gpg, etc. But it feels a bit clumsy, compared to the flake that contains the dev shell.
For my own system I set up home manager. But I don't want to make the use of home manager a requirement, as it can be quite opinionated. (e.g. setting up direnv will be done by generating a .zshrc, which can be limiting to some)
I think for people who don't want to dive into Nix much, doing an imperative install (nix profile install) of the necessary packages is also fine. You could even make your own small meta-package that depends on everything that is needed. Then they could do a nix profile install yourflake#yourmetapackage and have all the tools they need. But I agree direnv is a bit harder, since you'll have to put something in the shell rc/profile.
Thank you!
You frequently build things not to get binaries but to spend time compiling?
For example, sometimes I profiling the build times to see where I should focus effort.
Sometimes I want to see it to quickly check for issues where adding some dependency header causes build times to explode 100% in downstream dependencies during cold builds.
Another common occurrence for is trying to debug a platform, toolchain, or standard library issue and the build system either doesn't detect changes in those components or only makes the components readily accessible in an internal cache that's subject to invalidation issues. You'll usually get the wrong artifact or test results in those cases.
Some other systems (e.g. bazel/blaze comes to mind) actively try to hide side effects like stdout.
In all of these cases, the only way to actually get these side effects is to reach into the tool's internals by blowing away caches/output folders or reading live log files. That's a failure of the build tool.
Shake (https://shakebuild.com/) is pretty good about letting you specify that a specific step produces multiple artifacts.
I suspect Nix can do the same?
> Some other systems (e.g. bazel/blaze comes to mind) actively try to hide side effects like stdout.
Yes, blaze isn't all that great. You can tell, because Google folks check in generated artifacts into their repositories, instead of wrestling with getting blaze to build them.
It's definitely more work (you need to maintain compatibility with two different build systems), but worth it for exactly these reasons.
The main problem is that you often require more logic than makes sense to write in make, but it kind of has a language built into it so people try to use it. But as a language it's terrible (no scoping, many missing features). So people end up implementing their build logic in a bastard combination of make and shell which is very opaque and difficult to debug.
For example, I was recently trying to figure out how the OpenWRT makefiles are doing something, and it was really painful, because with make having no scoping any part of the system could end up affecting the piece you are looking at. There is a lot of dropping into shell to get stuff done, and a lot of the targets are themselves expanded variables, which makes it really opaque. Really a lot of it is not gaining from being written in make, they could do with rewriting large parts in a real language. But it would be a huge job. And that's where a lot of makefile systems end up
That's why you get tools like ninja where they decided not to allow any logic at all.
Or you get shake, where your logic is the logic of a real programming language.
That's actually not too much of a problem in practice: almost everyone just uses Gnu Make.
> Easy to write, easy to debug [...]
Alas, Make becomes hard to write and really hard to debug past a certain complexity threshold. And you reach that complexity threshold very quickly.
> Is it the syntax?
Yes, the syntax of Make is awful, and I'm not even talking about ergonomics. Thanks to Make's abysmal syntax, special characters in your files make it barf completely. And by 'special' I mean something as mundane as spaces.
But everything you mentioned is far from the worst. See eg https://news.ycombinator.com/item?id=17088328 for a more comprehensive overview of Make's sins.