Show HN: Test your Gitlab CI Pipelines changes locally using Docker | Hacker News Reader

Show HN: Test your Gitlab CI Pipelines changes locally using Docker

github.com

83 pointsdamnhotuser39 comments

Why not just use GitLab's runner[0] directly for this?

    gitlab-runner exec docker <job>

[0]: https://docs.gitlab.com/runner/install/

Yeah I thought about that, but gitlab-runner doesn't allow to "share" cache or artifacts between jobs so it's not a good fit to test multiple-jobs workflows.

Haven't test the tool yet. Neither do I know how close the behaviour of the pipeline locally will differ on the server.

Saying that, pipeline test tools are very much needed. Making commits and refreshing a UI is not practical.

I can test a distributed pipeline on spark why can't I do that with modern CI/CD?

Sure it does. Jobs have a shared cache (although this can be slow) and artifacts are shared to downstream jobs.

Maybe this is a tiered feature?

https://docs.gitlab.com/ee/ci/caching/

When using the "exec" command it doesn't do that. It just executes one job, not the whole pipeline.

exec is extremely limited and it's not being actively maintained.

As a GitLab / Docker user, testing pipelines is definitely a major pain point in my devops process. Cool stuff!

Nice! I made a similar tool a while back:

https://gitlab.com/rhn/cirunn

It's a pair of really simple Python scripts: cirunn.py for executing locally, an ciorder.py to execute stuff via ssh on another host (useful for projects that are too big for the laptop).

The main advantage is that podman does not need root access at all, making it both easier to use and more secure.

Also, no installation.

The difficulty I found here is deciding what the user actually means to run: is it more like `make test`, where the current state of the working directory is tested, or more like the CI pipeline, where only committed changes are tested?

For the sake of practicality, the local script tests uncommitted changes, and the remote one checks the last commit.

Having to push a commit to test the Github equivalent (Actions) was a huge pain. I think I ended up with 80+ commits.

I wish the existing solutions on the Github side worked.

Mildly ironic that you're hosting this on Github.

> I wish the existing solutions on the Github side worked.

I did experience a bad outcome using act[0] for a super weird matrix-y workflow, but I'd say for the most part it behaved sanely; did you report your complaints to their issue tracker?

0 = https://github.com/nektos/act

I briefly tried it. It couldn’t even parse the yaml.

> I think I ended up with 80+ commits.

Happens all the time to me. (although I have not used Actions) . Especially when I also had to set up a cross compiler on the CI. And each change needs 15 minutes to run the VM again

CI is an enormous productivity killer

Yep, CI like this is a catastrophe. Sourcehut's ad-hoc builds are really nice because you can test without pushing any changes. But sadly, Sourcehut CI is lacking in other areas.

IMO you can do this without needing extra tooling if you keep your CI scripts as shell scripts and your application is running in Docker.

If you keep your CI scripts as shell scripts and put them into a runnable file included in your project you can run your CI scripts locally so you can test the work flow on your machine. It's also handy in case CI is down you can still test and deploy your code. Lastly it lets you easily jump between CI providers since a majority of your logic is in a generic shell script.

For example in my CI specific files I typically only call `./run ci:install-deps && ./run ci:test`. Then there's a tiny bit of boiler plate in each CI provider's yml file to handle specific things for that provider.

If your app is running in Docker most of the heavy duty dependencies are all contained there. The dependencies I install directly in CI end up being things like installing shellcheck and helper scripts for CI (like tiny scripts that let you wait until a process is ready). Having WSL 2 is nice here because these are very small tools that I want locally installed anyways but even if you use macOS you could install these dependencies using brew instead.

A working example of this is here: https://github.com/nickjj/docker-flask-example

Check out the run script in the root of the project for "ci:" functions and the GH Actions yml file. This same strategy applies to GitLab or any other CI provider too.

We do this but with makefiles to contain whatever CI logic which also makes it easy to switch.

I also like this approach, although I am using Nix instead of Docker for a slightly more lightweight way of managing dependencies of different stages.

If you also reduce the assumptions made by your CI scripts, as in the scripts themselves will authenticate to and set up any needed connectivity / port forwarding, or fetch secrets instead of being passed secrets from the CI system, then you end up with some really nice portable scripts that can be run from anywhere.

The only things that my CI scripts depend on is an authentication token to Hashicorp Vault, an internet connection and Nix to be installed on the machine.

Same, my CI has looked like this for years now:

    ./script/prepare-env
    ./script/test
    ./script/build
    ./script/deploy

It's worked in virtually every CI system, and locally. Bonus point, it really points out the amount of ceremony some CI platforms require to map simple commands into their pipelines. Looking at you, CircleCI.

I found an approach like this was especially nice on teamcity in conjunction with its template feature.

Individual apps were still able to have whatever specific thing they needed, but you could globally add a step, eg a security scanning tool or slack notification by modifying the template only rather than 100s of build configuration files as the GitHub / circleci approach seems to encourage.

Adding a new pipeline basically just meant naming it and selecting the VCS URL

urgh no please don't wrap simple commands like docker build in shell called from makefile it's so f'ing complex to follow when it falls apart.

If you're going to do script-in-script like this at least go to a proper cross-platform language like python, powershell or js anything but make+bash it's just the worst of all worlds.

This looks great! I hate it when I have to commit and push a new change for a simple bug. That being said, how does this differ from something like act [1] (besides the fact this is for Gitlab)?

[1] https://github.com/nektos/act

> (besides the fact this is for Gitlab)?

Heh, yeah, besides being a different programming language, how is node different from Java? :-D

I, too, would value "the one CI yaml to rule them all" but given how much faster GitLab is moving than GitHub, there's almost no prayer of them trying to have some kind of common behavior.

I actually had the best experience using circleci's local runner (back when we were using it for CI) as far as "run binary, perform build" goes. I have yet to see gitlab-runner successfully execute a local build :-(

Thanks for a neat project! I tried it on one of our more complex pipelines and created a few issues for you ;)

How does this differ from Earthly? https://github.com/earthly/earthly

Earthly lets you abstract anything you do inside a container into an Earthfile, that runs locally on a dev's machine, but also in any CI, making CI scripts more portable. However, testing other CI-specific things before committing, such as GitLab rules dictating when jobs actually run, remain unsolved. But they also remain unsolved with this glci solution, right?

Interesting repo! It would be great to have more documentation since I am a beginner in Docker.

Not personal, maybe your unfamiliarity with Docker means you're not the target audience. I mean getting people up to speed with Docker is a bit of an ocean to boil, and this project appears to be small of resources...

Great idea. One question - what pain point is this solving?

As a GitLab, GitLab Pipeline and Docker user I don't find running pipelines that slow to run or initiate

Personally with Github Actions, I do find myself pushing 2, 3 or 4 times just getting the pipeline working because I always mess something up, either I've messed up a command or the environment isn't what I expected etc. There's act[0] for github which I should use more often to solve this, but I usually forget

[0]: https://github.com/nektos/act

You continuously need to push your changes to change `.gitlab-ci.yml` and find out if your change is working? E.g. your docker run command. I haven't tried out the tool yet but I hope it will save time and avoid me pushing 100s of commits to sort out my pipeline changes.

The problem is not necessarily slow init, but some pipelines are just long and complex.

If you have a problem in your deployment script, for example, you might need to wait a long time to test and fix it.

Why do you say great idea when you didn't even get the pain point it is solving?

From what I understand, you use it as a test for your pipeline scripts. With code we run unittest locally to make sure our commits are okay. When develop ci scripts, many times you only find out things don't work until you push the ci scripts to GitLab, it leaves you with many try and fix commits in history.

This looks awesome. I can’t wait to try it. GitLab CI is great, but tweaking it until its right is the worst part of my workflow. Thank you!

Great! I am currently working in an Azure Devops project. We would definitely need such a tool!

Cool, I'm gonna test this out. If it works well it's really useful.

This looks interesting. Will test it out. Hope it works with includes.

Yeah it does work with includes. One of the pain point right now is that it does not work with cache:untracked or artifacts:untracked, but I'm trying to figure it out :)

Haha, I have nothing that relies on that, so should be fine for me.

I guess you do something like secretly make separate artifact folders per job and keep copying (or just linking) stuff around?

any chance of running this without installing yarn?

Excellent idea!

Reply on news.ycombinator.com