Wikimedia Gitlab Migration Status
mediawiki.org
mediawiki.org
Gitlab CI is 3-4 different systems on top of each other wearing a trenchcoat. Every so often, they realise they can do CI better and come up with a new syntax for everything (see stages, which got extended by dependencies, which got superceded by the needs mechanism, or only/except which got replaced by rules). Obviously, they can't easily remove the old syntax, so pipelines very quickly become a mix of different mechanisms, some old, some new, with weird, unpredictable, and usually poorly documented interactions between them. You can try and only use a restricted subset of well-defined Gitlab CI, but part of the problem with modern DevOps is that most developers are not very experienced with the CI syntax, and will just find snippets on SO that do what they want and leave it there.
Apart from that, the documentation overall tends to be very poor, and the implementations are often buggy or missing functionality (a lot of searches for "Gitlab CI <thing I want to do>" would just link to an open ticket in the Gitlab repo describing exactly the functionality I needed, and dozens of comments going "yeah, this is necessary" and "a silver level customer needs this functionality to continue using Gitlab"). There's also lots of stuff that relies very heavily on being and to configure everything in YAML. other CI systems also have this problem, but often provide mechanisms to write individual tasks in other languages so you distribute, say, the "deploy this Docker image to our k8s cluster" task as a standalone unit written, tested, and reviewed in a real programming language. Gitlab CI has a very basic version of this feature, except it's still all in YAML, it's almost impossible to test, and it's very difficult to configure these tasks at all.
Fwiw, I've done plenty with Gitlab CI, and it's not like it doesn't work. It is perfectly fine if you don't have particularly complex needs, and the runners mechanism seems fairly easy for system administrators to get started with and get working. But overall, the whole system feels only half thought-through, and trying to do anything complex tends to require a lot of hair-pulling and confusion.
GitLab CI allows to run a bunch of commands in any Docker image you specify. The Docker image and/or your scripts may be self-written.
How do you find, for instance, GitHub Actions more advanced?
Being able to easily tap into an ecosystem of existing re-usable functionality (actions) is a great and pretty advanced feature that requires a very different (and more advanced) set of abstractions than running “a bunch of commands in a docker image you specify”.
Not that it’s a perfect system, but a core CI system with a decoupled layer of “things that run on that CI system” is a great model.
For example, the core of GitHub actions doesn’t have anything built-in that clones repositories. That’s a first-party action (component) that GitHub develops, releases and evolves independently. But you can roll your own if you want.
They've also introduced Components a year?(don't hold me to that) ago, which is more akin to the GitHub actions model.
> Avoid using global keywords in a component. Using these keywords in a component affects all jobs in a pipeline, including jobs directly defined in the main .gitlab-ci.yml or in other included components.
This is very different (and a whole lot less advanced) than being able to run a 3rd party GitHub action written in JavaScript, alongside another running in its own container image, mixed in with your own steps.
Because Gitlab CI bakes everything into one layer it’s much much harder to evolve and extend, and so it fossilizes whilst being unable to shake it’s baggage.
Roughly my issues with GitLab CI was that it didn't provide sufficient primitives for a) breaking up the build into a granular build graph, b) correctly passing artifacts between build graph stages (caching wasn't a good solution), and c) guaranteeing at-most-one task in a critical section (such as a deployment). It has been a few years since I last checked on these, but it also sounds like the product hasn't move much recently.
That’s extremely common now unfortunately.
> we believe that we should stop migrating all repositories unconditionally, and instead keep our two systems: Gerrit and GitLab. Gerrit should remain for the use-case of deeply connected repositories. GitLab should remain for tools, analytics and machine learning, and services.
> GitLab’s missing features are necessary for the productivity of developers, deployment safety, and operational reliability.
> There’s a demand for code hosting outside of Gerrit. Wikimedia Foundation-hosted GitLab has been a boon for these users—tool creators, engineers focused on data and analytics, and folks building services.
(of course, read TFA for the finer details and rationale)
Isn't this rather a symptom of code organization or architecture issues? If one needs to coordinate merges of individual parts, doesn't it mean they belong together? Monorepos or clearer versioning of the individual parts would enable independent merges.
Often I'm working on a set of changes that ultimately have a large code diff. I don't think huge PRs are often a great idea as they introduce more risk, so it's often a good idea to break it up. But a reviewer is not always available to review each bite-sized PR, so you end up with a backlog of PRs that are ugly to review in the GitHub UI and you need to continually rebase them as each is merged.
I quite liked the graphite.dev workflow for this, but it's a bit pricey. That and it only seems to work well if you can get your whole organisation to buy into using it. If GitHub etc integrated it as a native feature, I think that'd be great.
[0]: As mentioned in the article, and https://newsletter.pragmaticengineer.com/p/stacked-diffs
As far as I remember, and from what I observed, the decision to adopt GitLab was meant to better cater to newcomers and volunteers who generally do not appreciate Gerrit and saw it as a serious barrier to engaging with the Wikimedia software development ecosystem. Gerrit has a pretty steep learning curve and the web interface is pretty ugly (Subjective, but this is an opinion shared by many.) We got quite a bit of feedback that Gerrit was a stumbling block for new contributors as well new hires on the Product and Engineering teams. Many folks who have used Gerrit for a long time learn to love it but newcomers either hated it or found it difficult to adjust to.
So to summarize the main arguments for GitLab (as apposed to "just use github" or various other alternatives which were considered):
* It's ostensibly open * It's similar to GitHub in most ways that matter * The GitLab CI system is configured in the repo and it's entirely self-service, as apposed to the mess that is Gerrit + Jenkins + Zuul CI. Zuul requires a lot of specialty expertise to configure and maintain, and that places control of CI largely out of the hands of the people maintaining each repo. Self serve is better for the needs of many if not most developers. * Last but certainly not least, there was a fairly wide-spread fear that Microsoft would ruin GitHub, along with and a strong preference for self-hosted free software tools in keeping with https://foundation.wikimedia.org/wiki/Resolution:Wikimedia_F...
The two parts of my comment were unrelated, though.
> "Gitea Actions is still under development, so there may be some bugs and missing features. And breaking changes may be made before it's stable (v1.20 or later)."
Maybe this publicity will get them to rethink their strategy, but I doubt it.
I moved from a company using GitHub (and a decade of GitHub based open source experience) to a company using a Gerrit-like workflow, and I can see why people would be resistant to a move. On the surface the options look quite similar, but "at scale" I can see Gerrit working quite well and missing certain equivalents that would make a transition hard.
That's not to say GitHub/Lab aren't better, or that a move might not be worth it for other reasons, but a transition that pushes people out of their comfort zone and reduces their productivity for unproven, potential, future gains, is a hard one to justify.