Comment by mikepurvis | Hacker News Reader

Comment by mikepurvis | Hacker News Reader

Parent Full thread

Strongly isolated systems like Nix and Bazel are amazing for giving no-fuss local reproducibility.

Every CI "platform" is trying to seduce you into breaking things out into steps so that you can see their little visualizations of what's running in parallel or write special logic in groovy or JS to talk to an API and generate notifications or badges or whatever on the build page. All of that is cute, but it's ultimately the tail wagging the dog— the underlying build tool should be what is managing and ordering the build, not the GUI.

What I'd really like for next gen CI is a system that can get deep hooks into local-first tools. Don't make me define a bunch of "steps" for you to run, instead talk to my build tool and just display for me what the build tool is doing. Show me the order of things it built, show me the individual logs of everything it did.

Same thing with test runners. How are we still stuck in a world where the test runner has its own totally opaque parallelism regime and our only insight is whatever it chooses to dump into XML at the end, which will be probably be nothing if the test executable crashes? Why can't the test runner tell the CI system what all the processes are that it forked off and where each one's respective log file and exit status is expected to be?

> Write as much CI logic as possible in your own code

Nix really helps with this. Its not just that you do everything via a single script invocation, local or ci, you do it in an identical environment, local or ci. You are not trying to debug the difference between Ubuntu as setup in GHA or Arch as it is on your laptop.

Setting up a nix build cache also means that any artefact built by your CI is instantly available locally which can speed up some workflows a lot.

Absolutely. Being able to have a single `nix build` line that gets all the way from source to your final asset (iso, ova, container image, whatever) with everything being aggressively cached all the way along is a game changer. I think it's worth the activation energy for a lot more organizations than realize it.

Imo, the “activation energy” is a lot lower than it appears too. The determinate systems nix installer solves a lot of the local development issues and it’s fairly easy, as a first pass, to write a simple derivation that just copies your current build process and uses nix for dependencies.

Sounds like a business that surfaces the power of Nix with a gentle learning curve as a simpler, cleaner CI tool could have some success. Every time I see Nix come up, it’s described as very powerful but difficult to learn to use.

Dagger.io does this out of the box:

- Everything sandboxed in containers (works the same locally and in CI)

- Integrate your build tools by executing them in containers

- Send traces, metrics and logs for everything at full resolution, in the OTEL format. Visualize in our proprietary web UI, or in your favorite observability tool

It's doa, Sol.

Since you're using my first name in a weird and creepy way, I'll assume you're a hater. It's been so long since I've had one! Are you reactivating from the Docker days, or are you the first of a new cohort? That would be an exciting development, since getting haters is a leading sign of runaway success.

Not a hater - nobody will contest that Docker was a huge success, congrats... I just don't think Dagger has legs tbh.

Why would you need extra visualisation anyway, tooling like Nix is already what you see is what you get!

It’s still helpful to eg fold different phases in Nix, and different derivation output.

I work on garnix.io, which is exactly a Nix-based CI alternative for GitHub, and we had to build a lot of these small things to make the experience better.

Basically an online version of nix-output-monitor. Might be half an idea. But it doesn't get you 100%: you get CI, but not CD.

Delivery meaning the deployment part? I think by necessity that does differ a bit from happens locally just because suddenly there's auth, inventory, maybe a staging target, whatever.

All of that is a lot more than what a local dev would want, deploying to their own private test instance, probably with a bunch of API keys that are read-only or able to write only to other areas meant for validation.

We used to just tail the build script's output.

Maybe add some semi-structured log/trace statements for the CI to scrap.

No hooks necessary.

That works so long as the build script is just doing a linear series of things. But if it's anything remotely modern then a bunch of stuff is going on in parallel, and if all the output is being funneled to a single log, you can end up with a fair bit of wind-down spew you have to scroll through to find the real/initial failure.

How much better would it be if the CI web client could just say, here's everything the build tool built, with their individual logs, and here's a direct link to the one that failed, which canceled everything else?

> What I'd really like for next gen CI is a system that can get deep hooks into local-first tools.

But how do you get that sweet, sweet vendor-lock that way? /s

Reply on news.ycombinator.com