Nix is absolutely the best tool for what I want to achieve but it has those dark forsaken corners that just suck your soul out dry.
I love it but sometimes it feels like being a Morty on Rick’s adventure to the compilerland.
Things are straight forward on Linux. You build your binary, place in a docker container and you are done. The nix code will also be straight forward. If you can build your code, then creating a container is just one more operation away.
Unfortunately docker requires Linux binary and you are on Mac. So the docker desktop actually runs a Linux VM and performs all operations on it, abstracting this away from you.
Nix doesn't do that and you have two options:
1. Do cross compilation, the problem is that for this to work you need to be able to cross compile down to glibc, the problem is that while this will work for most community used dependencies you might get some package where the author didn't put effort making sure it cross compile. To make things worse the Hydra that populates standard caches that nix uses, doesn't do cross compile builds, so you will run into lengthy processes that might potentially end with a failure.
2. You can have a Linux builder, that you add to your Mac and configure to send build jobs for x86_64-linux to that builder. Now you could have a physical box, create a VM, or even have a NixOS docker container (after all docker ion Mac runs inside of the VM).
The #1 seems like the proper way, while #2 is more of a practical way.
I think you are running into issues, because you're likely trying #1, and that requires a lot of experience not only with Nix, but also with cross compiling. I wish Nix's Hydra would also build Darwin to Linux cross compilation as that would not only provide caches, but also help making sure the cross compilation doesn't break, but that would also increase costs for them.
I think you should try the #2 solution.
Edit: looks like there might have been an official solution to this problem: https://ryantm.github.io/nixpkgs/builders/special/darwin-bui... I haven't used it yet.
I'm using `clang` from `pkgs.pkgsCross.musl64.llvmPackages_latest.stdenv` to cross-compile Rust binaries from ARM macos to `x86_64-unknown-linux-musl`. It _works_, but every time I update my `flake.nix` it rebuilds *the entire LLVM toolchain*. On an M2 air, that takes something like 4 hours. It's incredibly frustrating and makes me wary of updating my dependencies or my flake file.
The alternative is to switch to dockerized builds but:
1) That adds a fairly heavyweight requirement to the build process
2) All the headache of writing dockerfiles with careful cache layering
3) Most importantly, feels like admitting defeat.
I pass the output of each stage of a toolchain as a dependency to the next stage. By chaining the stages, changes made to a single stage only require a rebuild of each succeeding stage. The final stage is the default of the flake, so you can easy get the complete package.
In addition, I can debug along the toolchain by entering a single stage env with nix develop <stage>
Not sure if this is the most optimal way, but it appears to work in modularizing the rebuild.(using 23.11)
Before someone asks. I've been using macOS for a long time. I've never seen remnants like this from a program. Sure, there are often directories left in ~/Library/Application Support/, but this was more than that. Unfortunately, I didn't write down the details, but I ran across the bits in at least 3-4 places.
- ~/.orbstack
- Docker context that points to OrbStack (for CLI)
- "source ~/.orbstack/shell/init.zsh" in .zprofile/bash_profile (to add CLI tools to PATH)
- ~/.ssh/config (for convenient SSH to OrbStack's Linux machines)
- Symlinks to CLI tools in ~/.local/bin, ~/bin, or /usr/local/bin depending on what's available (to add CLI tools to existing shells on first install — only one of these is used, not all)
- Standard macOS paths (~/Library/{Application Support, Preferences, Caches, HTTPStorages, Saved Application State, WebKit})
- Keychain items (for secure storage)
- ~/OrbStack (empty dir for mounting shared files)
- /Library/PrivilegedHelperTools (to create symlinks for compatibility)
Not sure what the best solution is for people who don't use Homebrew to uninstall it. I've never liked separate uninstaller apps, and it's not possible to detect removal from /Applications when the app isn't running.
And cough since we’re at with - did you consider Nixpkgs distribution?
I’m slowly moving deeper and deeper into ecosystem and use Home Manager for utilities that I use often (and use nix shell/nix run for one offs). Some packages are strictly GUI and while they aren’t handled flawlessly (self-updaters) it’s nice to have them on a single list.
Yet based on your list it’s a definitely a nixventure…
And yet, you are the only(?) one with that knowledge, so the alternative seems to be replying to HN threads with a curated list of things that a user must now open iTerm2 and handle by themselves. Something, unless I'm mistaken, that computers are really good at doing (Gatekeeper and privilege elevation nonsense aside)
Even just linking to the zap portion of your brew cask could go a long way since it would be the most succinct manifest if I correctly understand what it does
I agree this should be documented, but I still appreciate uninstallers.
Also, I'm a little confused about your statement:
> Not sure what the best solution is for people who don't use Homebrew to uninstall it.
You said at the start you've "been meaning to update the Homebrew cask to be more complete on zap" ... does that mean Homebrew uninstall will not do a complete job?
But unfortunately cross compiling quickly broke when I started doing mild customization (and one reasons I’m doing this is a complex setup that’s very sensitive to version changes).
In the end solution was to “simply” get darwin.linux-builder up but that pulled a lot of weight behind it.
It works, but it’s not the first time I spent my time on nix-ventures.
If something does break, rollbacks are free and an integral part of Nix.
This sort of twenty-minute adventure?
The rougher part is the speed of it; it's a one-two punch between the hardware & the fact that Docker has to emulate a Linux VM.
"I love it but sometimes it feels like being a Morty on Rick’s adventure to the compilerland."