Hacker News Reader

Comment by petethepig | Hacker News Reader

The biggest problem with Actions that I see a lot of people struggle with is that you can’t easily debug things when they aren’t going well.

I think they need to add some way for users to be able to ssh into the worker. Maybe each time an action step fails you have 5 mins to ssh into the container and inspect what’s going on.

View on HN

I use tmate[1] to get a shell on the worker when I need to debug things interactively. Also act[2] lets you run a decent approximation of your actions locally. Still agree that it should be easier and not require third-party tools/actions, though.

[1]: https://github.com/mxschmitt/action-tmate [2]: https://github.com/nektos/act

Yeah, the only downside of tmate is that you need to add/uncomment the line with it, push, then remove the commit again. It's much nicer in circleci, for example where you can simply rerun wish SSH in a button click. Even better would be if the VM was around for a bit after failing, so I could connect and inspect the state without rerunning at all.

One approach to avoid this karlicoss is to add a “workflow dispatch” section to the “on” events in the workflow:

  workflow_dispatch:
    inputs:
      debug_enabled:
        description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'     
        required: false
        default: false

Then under the job steps, make the debug step conditional on that optional flag:

     # Enable tmate debugging of manually-triggered workflows if the input option was provided
      - name: Setup tmate session
        uses: mxschmitt/action-tmate@v3
        if: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.debug_enabled }}

You then can just trigger the build on the required branch, setting debug_enabled to true and voila.

Edit: but agree the CircleCI interface is nicer where you can just run a debug build with ssh access directly.

oh, nice trick, thanks!

Thought I'd get their docs updated - https://github.com/mxschmitt/action-tmate#manually-triggered...

Yep, definitely agree. I feel like there are a lot of areas that GitHub Actions needs improvements to catch up to other CI providers, really basic-seeming stuff like "allow failure" support and so on.

action-tmate... woah!

I really like that with GitLab you can run the CI script in a local container to debug things and then only commit and push once it's working.

GitHub doesn't have exactly that, but it does have self-hosted workflow runners (which GitLab also has, as it happens.)

If you run a local self-hosted runner, point your workflow at it, and trigger it (push/etc), then the job bounces from GitHub's backend over to your local runner, where you can then poke / prod / trace its execution.

It's not quite as "safe" — you don't get to test the workflow in advance of deploying it, but rather have to test it by deploying it. But it's still pretty useful for debugging. You can e.g. set environment variables on the runner itself, that then propagate into the job, to see how they affect the job. Or change stuff in the runner's user's home directory until the job works, to figure out what the job needs there, rather than deploying over and over with different "tweak this file or that" steps.

That's an even better solution.

Post-fail ssh access is one of the headline features of https://builds.sr.ht

CircleCI lets you rerun a pipeline with ssh enabled so you can debug. It's helpful unless your failed step is at the end of a complex 20 minute deployment. I'll usually run my docker image locally, copy over the source, and run the pipeline by hand to debug, but then I have to set up all the environment variables that I get from CircleCI/Bitbucket Pipelines, and even then there's no guarantee that my local tests are identical to what I'm seeing in the pipeline (just today I hit a memory limit in Bitbucket pipelines that I couldn't recreate locally).

https://tunshell.com/ works great for *nix runners.

I made https://github.com/nelsonjchen/reverse-rdp-windows-github-ac... for the Windows runners.

Just add continue on error to the failing step and use temporarily use these kinds of tools in the next step(s).

> https://tunshell.com/ works great for nix runners.

Oh and their interface is a `curl | sh` that execs whatever happens to be lying around in `/tmp/tunshell/client`. I guess* that is probably fine on a single-user system but I prefer not to run binaries that just about every process on my system can write to.

https://lets.tunshell.com/init.sh

Edit: I filed a bug https://github.com/TimeToogo/tunshell/issues/19

> I guess that is probably fine on a single-user system

Most of these services run your workload inside a container, so it's not even a single-user system; it's an ephemeral system. It's like anything's going to be there other than the base image + what you put there.

This same script is used for the server and client. So you are running this on your workstation.

I'm not that familiar with GH actions, but Sourcehut has a couple of pretty nice solutions for troubleshooting: (as you mentioned) time-limited SSH into the build server and a page to edit the manifest online and re-run without having to push anything.

You could create a self-hosted runner to test things out on. I'm not sure how hard it is to match Github's VM configuration but I don't think they veer far from the stock ubuntu config for instance.

I've used https://github.com/nektos/act in the past. It works ok as long as you don't need artifacting.

Yea setting up Actions is a pretty awful experience. I had to constantly git add, commit --amend, force push and retrigger the run and look at print statements.

I mostly struggle with the docs to be honest.

But yeah, SSHing in is great. I miss that from Circle using GL & GH.

The ability to ssh into the Travis VM and investigate the terminal state on failure is just amazing.

Reply on news.ycombinator.com