Comment by scrollaway | Hacker News Reader

Comment by scrollaway | Hacker News Reader

Parent Full thread

This stuff sounds incredibly exciting.

Replacing Jenkins has been... a frustrating experience met with nothing but hollow shells of services. We want to self-host, we want something that can deploy our code, we want to be able to write arbitrary jobs not tied to specific github projects.

The Jenkins ecosystem is atrociously bad. Don't get me wrong, it's incredibly active, but it promotes a "just make a plugin for everything" attitude which results in things like, plugins to echo this or that, plugins to do a million things better done from the shell, etc. On top of that, the Jenkins UI is just... bad. It's ugly and it's a UX disaster. I could go on.

Yet, there's nothing better than Jenkins out there right now. Drone claims to want to replace Jenkins, but doesn't even have support for arbitrary jobs. I'd love more recommendations.

This is why, a while back here on HN, I asked if I could pick up the Gitlab job runner in golang and slap a different web UI on top of it that manages the jobs. This doesn't appear to be possible but I see the value in such a runner existing. Running the jobs is a critical component of such tools.

I just finished setting up a complete pipeline for a new business with my cofounder: Jenkins was a big pain point. Terraform was a pain point. Ansible was a pain point. Getting a vault solution set up and done right was a massive pain point. We've been striving to make things as simple as possible for developers so I wish you the absolute best of luck in what you're doing, I'm continuously impressed with Gitlab.

Email me if you have questions Sytse, I'd love to give more feedback -- email on my profile.

We don't like plugins because they are a bad user experience, make it hard to upgrade, and make it hard to improve the product. We rather see people contribute features to GitLab itself including tests. This way we can make sure they are nicely integrated in the UI and that they still work when we release a new version of GitLab. This allows users to upgrade with confidence and it allows us to make major changes to GitLab.

I'm not sure what you use case it but GitLab CI is pretty extensive and I hope it has everything you need. Please let me know if there is anything missing. If you want to host your code somewhere else you can still use remote mirroring to have GitLab do the deploy.

The GitLab Runner is a simple piece of software on purpose. This allows us to port it to many platforms. All the logic is in GitLab itself so it is easy to provide a lot of feedback about the status in the UI.

The problem with Gitlab CI, last I looked at it, was the inability to untie it from gitlab and run+self host it standalone. Has this changed?

I would love it if Gitlab CI went back to being a separate product. I'll use Gitlab for my own projects but ones that I work on with others are challenging to migrate. And creating mirrors on Gitlab of repositories hosted elsewhere to make use of CI/CD features doesn't sound ideal.

I agree it is not ideal but it should work pretty smoothly, have you tried https://docs.gitlab.com/ee/workflow/repository_mirroring.htm... ?

I understand why your different offerings are bundled together, I appreciate how your team tends to reach out to users in public forums such as this one, and I'm a fan of the quality of your work, but quoting from the page you linked to:

"any commits should be pushed to the upstream repository, and will end up in the GitLab repository automatically within an hour, or when a forced update is initiated."

"Wait up to an hour for things to happen" isn't a great story to pitch to team members :/ I'm also talking about at least twenty different repositories. I'll most likely have to continue using Jenkins for those projects but I look forward to using Gitlab CI on weekends.

I thought this time was configurable if you run your own installation of GitLab EE. I agree that for GitLab.com the delay of an hour can be a problem. Maybe we can solve that by adding webhooks?

Yes, webhooks would be preferable over polling.

Cool, I created a feature proposal for this https://gitlab.com/gitlab-org/gitlab-ce/issues/22139

Awesome, thanks!

The only way to use GitLab CI is as part of GitLab. By doing this we could deeply integrate it into the interface of GitLab and make it a better user experience. If you host your code anywhere else you can use remote mirroring (EE/.com only) to use GitLab CI.

You can't make it replace Jenkins if it can't do arbitrary jobs. I'm talking jobs without even a repository behind them.

The idea of a runner that tests every commit on a repo is super useful. But Jenkins goes one step further by being essentially an arbitrary shell command runner. I use it for example as a CNC for running cleanup jobs, cronjobs etc.

Maybe the repository can contain the code of the cleanup jobs?

This is shoehorning something into a model that doesn't fit it :/

For example, on Jenkins right now, we have a job which can trigger a bunch of load tests against our website. We can run this job indefinitely, give it arguments, it'll even run a local web server. By default, the job only runs the load test for ~10 mins.

The job is run periodically. When it fails, we get emails. After it completes, regardless of failure, it triggers a "cleanup" job which deletes the loadtest-related data from the website.

This is what I mean with "arbitrary jobs". I think it's absolutely fair to consider them out of scope, but if you want to replace Jenkins, this is what you're going up against.

Thanks for sharing your use case. I think that code of the cleanup job can be in the same repo as the load test itself. Would that be a fit for you? Of course the load test code can have the testing code of the application as a dependency.

Maybe you prefer the load testing code in the same repo as the application. In that case it should be two jobs that are triggered periodically I think.

I think what GitLab CI still needs is periodic jobs https://gitlab.com/gitlab-org/gitlab-ce/issues/2989 (right now the workaround is sending a build trigger for an external task).

Can Gitlab CI do manual triggers yet? (Drone can't, last time I checked, and that was a huge turnoff)

The "code" in this case really lives in the Jenkins database, as it's a shell snippet. Admittedly I'm not a fan of this. However, since you're a FOSS company as well, you may have hit the case where you want to separate the code for your infrastructure from the code you're releasing in the general repository, but still build off master.

This is where Jenkins' model is quite good. I strongly dislike that it's so hard for Jenkins' metadata to be versioned in proper git repositories, but the separation of the actual code, and the CI-related code is great.

Manual build triggers for GitLab CI are possible with https://docs.gitlab.com/ce/ci/triggers/README.html

In my taste the shell snippet that is infrastructure specific should like in a small repository. An example of that for us is https://gitlab.com/gitlab-com/runbooks (although that is more process since we try to keep the code generic).

no, it does not fit the bill. It needs truly arbitrary job running.

I want to be able to just drop in some arbitrary command (e.g. df -h /some/build/agent/dir/ | grep 99%) that would not be worth it to me to write a script, commit it to git, get that deployed and running.

In fact, I AM a fan of this being in git, but the example I'd use is an improvement on TeamCity, where you can configure your job and project configurations to "commit to git" so that you can easily restore when you need to rebuild a server or reload a previous version.

Sooooo many use cases for arbitrary jobs. Many of which may be short lived, but want them automated in some way.

(speaking as a sysadmin/eng/devops/cloud dude)

The new version of drone (0.5) is much better imho, although it is still in beta so can be a bit frustrating at times. They have relaxed a bunch of the constraints and removed alot of the magic. Plus plugins are alot easier to write now.

(Not involved, just a happy user)

  > I'd love more recommendations.

I have been using buildkite lately, and it has worked rather well. You run the agent on your own hardware, and buildkite itself just acts as a coordinator and log collector.

Reply on news.ycombinator.com