Hacker News Reader

Comment by Nextgrid | Hacker News Reader

The way I deal with all these terrible CI platforms (there is no good one, merely lesser evils) is to do my entire CI process in a container and the CI tool just pulls and runs that. You can trivially run this locally when needed.

Of course, the platforms would rather have you not do that since it nullifies their vendor lock-in.

View on HN

I really like the SourceHut CI, because:

1. When the build fails, you can SSH into the machine and debug it from there.

2. You can super easily edit & run the manifest without having to push to a branch at all. That makes it super easy to even try a minimum reproducible example on the remote machine.

Other than that, self-hosting (with Github or preferrably Forgejo) makes it easy to debug on the machine, but then you have to self-host.

Self-hosted runners with Github is a whole world of pain because it literally just runs commands on the host and does not handle provisioning/cleanup, meaning you need to make sure your `docker run` commands don't leave any leftover state that can mess up future/concurrent builds. It doesn't even handle concurrency by itself, so you have to run multiple instances of the runner.

That was included in my "but then you have to self-host" :-).

As I said, I really like the SourceHut CI.

Do you have any experience self-hosting SourceHut? I’d really like to do so, but I get weak knees every time I look at the docs for it.

I don't, but I would be curious.

But I'm happy to contribute (money) to SourceHut, they're doing a good job.

Github supports ephemeral runners which are limited to a single job.

You can use `workflow_job` webhook to be notified of a new job, after that you need to call `generate-jitconfig` API to get a just-in-time configuration token, and then you can start a Github runner in ephemeral mode with the JIT token.

This allows you to orchestrate Docker containers, KVM instances, etc., which are used for a single time, and then destroyed.

There are some open source projects, such as using ephemeral Kubernetes pods with the ephemeral runners.

Forgejo handles all of this quite well and is, in my experience, muuuch faster than GHA

Thats what i always did for our gitlab CI pipeline, just deploy dedicated images for different purposes. We had general terraform images for terraform code, this made it easy to standardize versions etc. Then we made specific ones for projects with a lot of dependencies so we could run the deployment pipeline in seconds instead of minutes. But now you need to maintain the docker images too. All about trade-offs.

The one issue with that is there isn’t a good way to containerise macOS builds.

I mean this has been an issue building for iOS forever. The MacOS lock-in sucks really really badly.

It's kind of their whole thing, when you think about it. They didn't get to where they are by playing nice with others. If you're supporting anything in the Apple ecosystem, the fix is in.

Github delivers an exceptionally detailed webhook. Ironically, there is zero actual lock in, the CI/CD cargo cult is that strong they didn't even need it.

Your newsletter. I needs it.

Reply on news.ycombinator.com