Hacker News Reader

Comment by simonw | Hacker News Reader

If you want the ability to run as much of your GitHub Actions locally as possible (which you should! Testing via pushing commits is painful) you can arrange your actions so the majority of the logic lives in shell scripts (or Python scripts or whatever) that are called by the workflow.

Then you can run those scripts locally and iterate in them before you try them on GitHub Actions.

I agree: Actions would be more useful if I could run the entire stack locally (an official implementation, not a community-maintained clone). But it's not a big enough inconvenience for me to care very much.

View on HN

The problem is that GitHub Action offers a large ecosystem of reusable actions that you can't easily call from a script.

I mainly ignore those - I stick to the actions that install things (setup-python) and run scripts.

You can get around the setup ones by using Docker images (e.g. run in python:3.11 rather than using setup-python).

Can you provide a link to an example?

The docs have examples: https://docs.github.com/en/actions/using-jobs/running-jobs-i...

Basically you set `container.image`.

You should check out: https://github.com/nektos/act

Works pretty well for the actions I have used it on.

That's not a problem. You just don't use them if you don't want to suffer vendor lock.

> That's not a problem. You just don't use them if you don't want to suffer vendor lock.

How do you use GitHub Actions without using actions such as actions/checkout, actions/upload-artifact, actions/download-artifact?

You, you sacrifice that convenience for something purely in shell that is more portable.

you you?

It's like a yo-yo but faster

Like yo-yo, but slower due to the extra W.

This is the way. It also lets you pretty much seamlessly switch CI tools at any time.

I third this opinion: another reason it’s important is it allows you to do your build in an emergency when CI is down for whatever reason.

CI automation is for me the thing that replaces me running scripts one by one (and reporting/deploying results)

It is not the thing that does the building/testing/deploying. That’s the scripts that are hopefully written in a debuggable, portable, language.

This is a really good point. Most of my deploys go via GitHub Actions these days and it's not easy for me to deploy manually should I need to when Actions is unavailable - I should fix that!

Like aviation, every regulation/rule is written in blood ;)

This is why the three of us should record a podcast episode talking about this in the context of the points made in the Ironies of Automation

https://en.wikipedia.org/wiki/Ironies_of_Automation

Agreed - I had to switch from TeamCity to GitLab earlier this year and I was very glad that all of pipelines were single-line calls to shell scripts.

Indeed, to rely on third party actions in a closed runner ecosystem, is simply unwise.

Isn't that exactly what the self hosted runners are?

https://docs.github.com/en/actions/hosting-your-own-runners/...

Then you configure the repo settings to only use the self hosted ones.

If you're suggesting the self-hosted runners are equivalent to running locally, then no, self-hosted runners still require a commit + push and runs remotely with the added overhead of the job being queued, reported, etc.

The dream is to have something like a container _locally_ and be able to run something like:

    > gh actions run

I wish I didn't have to circumvent the UI with a strategy like this. Part of the reason I like CI/CD tools is the visualization factor - I can create specific steps, see which step is currently active during execution, and only care for the output of a specific step during debugging.

A platform with support for visual control from the scripts (implemented as no-ops during local execution) would be perfect.

If you split your scripts into small tasks (i.e. one task per script), and call each one from your GA yaml, you would seem to have the best of both?

That still makes running it a bit annoying, since you have to call each script in order. You could of course create a script that covers this, but that's duplication that can drift apart. I guess you could generate the script from your YAML or use a linter or something, but it all feels a bit annoying. It would be awesome to e.g. just call a command like "gh-success" or "gh-failure" to manually indicate the success/failure, maybe with an optional ID to prevent accidental double calls etc.

Doing this also means that your build scripts are version controlled. You can iterate on the build scripts in different branches, for example.

GitHub actions are configured by version-controlled files too though. I have an actions-dev branch on my project, so I can iterate on actions config without having to clutter up master.

Would still be better if I could run locally to iterate. But the local run would have to have very high fidelity to what happens on GitHub to be useful.

Running actions triggered from a non-main branch does it for me, either on pushes to the branch, or 'workflow_dispatch'. You'll otherwise never get that level of fidelity to what happens in production.

Keep the logic in your YAML "dumb". Avoid variables and subroutines in the file, if you want to DRY something, create your own custom action. You can have unit tests on an action, you can't have unit tests on a GHA workflow.

That cuts both ways, in my experience: a bug fix needs a metric boatload of cherry-pick, possibly across a bunch of repos, to fix any bugs. Centralization is great until it's not

We have gotten a lot of good mileage out of GitLab's include: feature <https://docs.gitlab.com/ee/ci/yaml/#include>, which for clarity absolutely carries this same hazard, but can mean centralized fixes are less onoz than trying to patch every active branch of every active repo's build scripts

I am also aware this cuts against the thrust of the thread about "build scripts should be locally executable," but until they (they: GitHub, they: GitLab) straighten out their runner binary to be sane, it is currently a better use of our engineering effort maintaining CI yaml than teaching devs how to run CI locally

Incidentally, this is sort of how the Earthly runner serves to provide you local actions that also work in CI.

I'm not quite familiar with GitHub Actions, but Azure DevOps Pipeline has a nice Preview API: https://learn.microsoft.com/en-us/rest/api/azure/devops/pipe... which runs the preprocessing steps(like running the C preprocessor on a C source file) then gives you the processed yaml file. Then if you'd like, you can write your own local runner based on that if your yaml files do not use too many different kinds Azure DevOps pipeline steps.

yes! what we did was use a task runner to encapsulate the work in a “ci” task. you can run it locally, and then the GHA runner does little more than clone, install a couple depa and run the task.

This is the way.

Building Java and C# on the command line nowadays is a hard undertaking, requiring weeks of learning and debugging, just to have your code change in a way that makes the scripts outdated at any time (oh, and it's never portable).

I really don't get why people use those complex CI/CD tools for other languages, but at least on the enterprise ones, I can understand people moving from the IDE into some huge centralized mess that can import it.

Why does Java take weeks?

For a straightforward application or library project, you can fill in this form and get a shovel-ready build: https://gradle-initializr.cleverapps.io/

I usually tweak it a bit, but that's minutes, not weeks.

There's often more stuff to add as a project grows, but again, minutes to hours for the common stuff. Setting up publishing to a Maven repository is more work than I'd like, but it's still not weeks.

So what are you doing that it takes so long? And how long would that take if done any other way?

Building C# is quite literally 'dotnet build -c release'.

Exactly. I don't know where the newer .NET versions get the bad rap from.

You can criticize Microsoft for other things but .NET is not one of them. They've done a great job with it.

(And building code that targets the classic .NET Framework can usually be accomplished by a single call to msbuild.exe.)

Reply on news.ycombinator.com