Comment by qbasic_forever | Hacker News Reader

Comment by qbasic_forever | Hacker News Reader

Parent Full thread

Check out tekton CI (https://tekton.dev/), it's a Kubernetes operator to run a CI pipeline that's defined as commands running inside any container. Use any language, any commands, etc--as long as you can get a container image, you're good to go. There's a growing set of community created and curated actions to do common things too: https://github.com/tektoncd/catalog

Yeah you need a k8s cluster, but even a simple kind dev cluster that you spin up in 30 seconds with one command on your laptop will work.

I like it a lot because it enforces very little structure on you and doesn't reinvent everything. Stuff like storage (either ephermeral or existing volumes), secrets, configuration, etc. are already modeled and supported by Kubernetes and tekton can use all of that natively. And since it's all k8s native stuff you have all of the power of k8s, like its entire API for manipulating and managing execution, exposing services, etc. There is very little cognitive overhead or new things to learn once you know k8s.

If you're really averse to k8s though, check out drone. It has a local execution mode that is similar and just runs whatever pipeline commands you want in docker containers. https://github.com/drone/drone Batect is another even more minimal tool that's effectively just a docker-based workflow system: https://github.com/batect/batect

We use Tekton to manage our CI pipeline and I agree that the way it enforces very little structure is a strength. On the other hand it's new enough that if you need to stretch its capabilities you are going to have to get creative. The primitives it has are nice, but they have their limits.

For instance, if running a bunch of parallel tasks, collating results on a PV is out the window unless your cluster supports multiple writer volume types, which GKE does not. You have to bring in NFS volume types or something like that for it. In the early days of tekton they had a results primitive which synchronized an output dir to GCS, but they decommissioned that. So you are left pushing that logic into your task command. Running gsutil is easy enough, but it means you are pushing logic into your scripts and not declaring steps in the pipeline definition. You could make that command a step but I see little benefit in that.

Additionally there is no way to loop in the configuration to generate tasks, much less loop with an ordinal value. We end up just programmatically generating the resource definitions with ruby erb templates. All of our pipeline specs (including task runs, etc) creates a 2MB yaml file. We push dozens and dozens of these into k8s daily. It works but at the same time our usage of Tekton is more or less as a glorified alternative to batch jobs which works because batch jobs _still_ don't have a proper sidecar capability and also because we rely on the DAG to order dependent taskruns.

If your pipeline is simple, look at Tekton. But if your pipeline is complex... still look at Tekton but expect to do some work. Once you get a good workflow though you can you can scale your pipelines as easily as you can a deployment in k8s. We use node autoscaling and preemptibles (Tekton can retry if a task disappears due to node reclamation) to manage our CI costs quite effectively.

> Running gsutil is easy enough, but it means you are pushing logic into your scripts and not declaring steps in the pipeline definition.

Some other comments here have argued for pushing as much logic as possible into your scrips, so that they can be tested without the CI system. What's the downside of doing this?

I’ve been meaning to give Tekton a try, but how is the visualization (pipeline graphs ets) and Github/Gitlab/CI integration for returning results and viewing build output?

There's a super basic dashboard: https://github.com/tektoncd/dashboard Tekton is really more of a lower level engine for CI though and doesn't focus on building UI, etc. You might check out jenkinsx which uses tekton but builds a whole new jenkins experience on it: https://jenkins-x.io/

Reply on news.ycombinator.com