Comment by codetrotter | Hacker News Reader

Comment by codetrotter | Hacker News Reader

Parent Full thread

Yeah. We have a similar thing going on with self-hosted GitLab at my work. (And there, there isn’t even any incentive why it should be slow.)

When I make an edit and rebuild on my computer, it takes a few seconds to rebuild because most things are cached.

In CI it takes 20 minutes on the MR branches of our repo. 30 minutes on master. Because a whole bunch of crap is downloaded from scratch and rebuilt from scratch.

Our infrastructure guys did set up something that is able to cache stuff that some of the repos use. But with the limited access I have I have not been able to figure out how to make use of it for the repo I work on. And they don’t have time to look into it for me either.

That’s my life lol

You can define a cache key in your .gitlab-ci.yml

https://docs.gitlab.com/ee/ci/yaml/index.html#cache

Considering gitlab is a very k8s native app it's way of running a build and moving caches into it, vs moving the container over to where the cache's are is very annoying. I've had issues open for years to update the cache-key with a CSI mount and let k8s move the pods to where the mount is fast. As it is now it pulls everything from S3.

Yeah gitlab has pretty good caching support and it is basically a must if you do bigger jobs, especially on stuff like c++ apps. It is pretty brittle sometimes, and in my experience it won't be able to be as fast as actual local dev but it's still good. You can even make it use a remote cache IIRC.

On free runners I'd regularly hit the 1 hour limit. Took some time to set up self-hosted through EC2 and really optimize caching, its now down to 10-20 mins, depending on the repo. Most of that is because of the self-hosted runners, the free ones are painfully slow.

Still, on my local machine these jobs would take 5 mins so its not perfect. And as the build gets more complicated and more stages are added, the problem compounds since the initial download is the slowest part.

Reply on news.ycombinator.com