And that's actually pretty sad, because Gitlab is the only open-source alternative that can keep up with Github feature-wise.
I think it's a bit the opposite, GitHub seems to have started adding most of these features as a response to competition with GitLab. Not that GitHub hasn't pulled ahead in some ways now.
We'd personally prefer to see a better release when it's done, vs. them keeping this farce of a 'release streak' going and then asking customers to install another upgrade within days.
I don't have any particular opinion of Gitlab, but it does seem to be that acknowledging fault is more valuable than the alternative. If I were to attack a service, I'd probably tend to avoid the one that actively updates it's security regularly.
It is an age old conundrum that people seem to struggle with a lot. As part of my previous profession, I've security reviewed hundreds of open source libraries. They fall into 3 categories:
1. The libraries/applications that have either had a really security conscious developer behind it or a very rocky past such that it was given an extraordinary amount of attention security wise. These account for about 1% of software
2. The "normal" libraries/applications that nobody care about with regards to security. As long as the code works everyone keeps using it. They are often insecure by default, but the lack of attention by SME means they wont be judged. They account for 80% of software.
3. The horrid ones. Built-in code execution as a feature. Authentication systems with more critical bugs that you can possibly image. We never talk about these because... well, we could spend months on polishing a turd. Nobody dares to publicly speak up and say "don't ever use X, Y and Z" for fear of the repercussions. They account for the rest of software.
GitLab used to be in category 2, but got moved into 1 about two years ago when security professionals started to give it attention.
As to your feeling, 99% of software have security issues, some worse than others. We rarely talk about them.
I’ve been running a private instance for about a year and am absolutely in love with it. Gitlab CI is the killer feature IMO, and self hosting it means I never have to worry about usage limits.
But if all you need is basic git hosting with an issue tracker, I don’t see a reason to use Gitlab over something like Gitea.
How does gitlab.com deal with this? Or do they just put up with rando users signing up and spamming the snippets/issues/etc.?
We have internal tooling that we're working on incorporating into GitLab itself to help with this: https://about.gitlab.com/blog/2021/08/19/introducing-spamche....
When will it ship?
https://docs.gitlab.com/ee/user/admin_area/reporting/spamche...
I'll have a look.
> Just because I don't think I said it explicitly anywhere above: Because we are using an obfuscated, non-free component (the preprocessor), we can't include spamcheck in CE (users of CE expect no proprietary code to be included in the pacakge), but only in EE.
So... is it available in the current version of gitlab-ce or not? I don't want to waste time trying to get it running only to find out you've only made it available for enterprise editions and gitlab.com.
1: https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6259
You can run EE without license, it provides the same features as CE. Maybe that is an option for you: https://docs.gitlab.com/ee/update/package/convert_to_ee.html I've created an MR to help clarify the docs: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81751
Anyhow, this sounds like a death knell for gitlab-ce. My GSoC use case isn't fringe (there are 100s of GSoC orgs), and Gitlab wouldn't have spent money on the ML approach for EE if it weren't generally important.
EDIT: someone on that issue summarized the issue pretty well:
> Its really annoying how fragmented gitlab is rightnow. I have a dozen accounts on a dozen instances. This feature combined with oauth login to other instances, would make it like there is one big gitlab we all use!
In particular, if an instance or specific repository concerns only people from specific projects/instances, it would be easy to allowlist those specific instances and not have to deal with spam at all.
And most - if not all - potential GSoC contributors are from gmail addresses. So again, I don't understand how this could be a general solution to spam.
- a new organization using your project? maybe grant their whole gitlab instance "issues" read/write access to the project
- publishing FLOSS in a "community" setting where random people submitting contribution is not expected? maybe we can check the PGP WoT before deciding whether to accept that PR
- running a federation of organizations, some of whom may run their own instance? allowlist all the instances so they can interact across instances
- running a public forge like gitlab.com, codeberg.org, or chapril.org? maybe maintain an allowlist of servers who ask for it and pledge to fight spam
- feel adventurous? setup an entirely public instance and help catch spam and reporting it to denylists
All this is already possible on email level, but pointless as you pointed out as trustworthiness of the mail server is not correlated to trustworthiness of the forge.
You can disable signups, or require all users to being approved by an administrator, if that works for your instance. https://docs.gitlab.com/ee/user/admin_area/settings/sign_up_... There are more ways, like limiting specific domains for signup.
Future spam detection ideas shared in https://news.ycombinator.com/item?id=30479511
> You can disable signups,
I want potential GSoC participants to be able to sign up.
> or require all users to being approved by an administrator, if that works for your instance.
I don't have time to hand separate the Indonesian casino spammers from the potential GSoC participants. And I do mean "by hand"-- the Gitlab UI requires me to click a button to open up a secondary menu, then choose add or delete, then wait for the user screen to reload.
At least when sifting through an email spam folder back in the 90s I could press the delete button multiple times in a row. Even that would be a relatively usable solution.
I made some scripts to do this, but would have to extract them from beside the user data in the repo.
Imo they're still leading Github in some areas. My favorite Gitlab exclusive feature is a very simple one: folders inside of projects. Usually Github has followed a pattern of catching up to Gitlab, so I think it's probably only a matter of time before they add this feature (if they haven't already, it's been a while since I checked).
Actually I take that back, my favorite feature is that I can host my own private instance. Gitlab gets even better if you have admin access to the system. And that's one feature Github will probably take a while to copy, if ever.
Eh you can already self host GitHub, longer than GitLab has been around, it’s just not free to do so.
My general rule of thumb is that if a product lists its price as "Contact our sales team", then its effectively unavailable to me as an individual. So I guess if we're talking about what exactly the Github "feature" is that MS won't copy, it wouldn't be that there's an enterprise option, but that Gitlab is free and open source and practical for me as an individual to install. Obviously if you pay Microsoft enough money they'll do whatever you want (up to and including I guess buying Github itself as a company from them. Always has been an option, it's just not free to do so.)
$21 per user per month.
Interestingly, this page isn't linked to from https://github.com/enterprise (except in the footer), where the calls to action are to either "start a free trial" or to "contact sales".
Why does this feel like people are trying to turn this into some sort of gotcha? It wasn't even my main point. The point is that GitHub enterprise is not an option for individuals.
Original comment from you:
> Actually I take that back, my favorite feature is that I can host my own private instance. Gitlab gets even better if you have admin access to the system. And that's one feature Github will probably take a while to copy, if ever.
I never claimed it's "priced out for every single small team or individual user ever". You even quoted it right there:
> my favorite feature is that I can host my own private instance.
I have 33 people on my personal Gitlab instance. It would cost me over $8k per year to run a Github Enterprise instance with that many users and I don't have the kind of money to do that. Until I can do that with Github for $0 then my point stands.
They're both commercial companies chasing dollars. The OSS sales model is simple. Get companies/people hooked on OSS, have them reach the boundaries of the OSS product and then sell them a commercial license. Most people can and will just move to the better enterprise offering which is probably why I see so little GitLab out in the wild. Based on what I'm seeing/hearing from MSFT, GitHub is basically a free product that now comes with your MSFT enterprise agreement. If you agree to spend enough Azure or Visual Studio dollars MSFT agrees to pony up GitHub licenses. GitLab can't compete on that level, and the offering alone isn't compelling enough for your average fortune 500 when it comes to spending money.
I enjoy GitLab and it’s the real only option for open self-hosting, but it made me miss the days of having GitHub Enterprise.
I switched to Gitea a long time ago and have no regrets.
In the past, you could go from current version -> latest z release -> X.0 release of the next major -> latest minor release of Z. As long as gitlab completed the db migrations and came up, you were (relatively) ok to continue to the next upgrade.
Turns out 14 introduces a brand new upgrade failure case. Starting with 14, upgrades can include async db migrations that must complete before you continue upgrading. But once you start upgrading, it's too late, things are hosed, so now I'm falling back to a backup and starting over.
Thus, it might be a better idea to look into a less popular stack that's less likely to be targeted as much due to not being such a juicy target.
For example:
Code: Gitea/Gogs/GitBucket
CI: Drone/Jenkins (okay there are probably better options than Jenkins, to be honest)
Registry: Nexus/Artifactory (not just for containers, they support most formats and have better control over cleanup of old data so you don't have to schedule GitLab cleanup yourself)
> It seems you are upgrading from major version X to major version Y.
> It is required to upgrade to the latest Y.0.x version first before proceeding.
> Please follow the upgrade documentation at https://docs.gitlab.com/ee/update/index.html#upgrading-to-a-...
In addition to that, you should NEVER just bump versions without having backups (which you've hopefully considered), so there is probably another step in there, either validating that your latest automatic backups work, or even just manually copying the current GitLab data directory into another folder, in the case of an Omnibus install, or doing the same manually for all components in the more distributed installation type.
Disclaimer: this has little do to with GitLab in particular but is something you should consider with any and all software packages that you upgrade, especially the kind with non-trivial dependencies and data storage mechanisms, like PostgreSQL. Of course, you can always dump the DB but it's easier to back up everything else as well by taking the instances offline and making data copies of all container volumes/bind mounts.
* I have automated backups created every 2 days stored via S3, I've done a full restore twice in 5+ years of uptime.
* I run Gitlab at home and at work.
None of the points touch on a maintenance burden ... Just saying. Skipping versions while updating any software is just being a lazy sysadmin and praying it works. Typically skipping to major versions during upgrades always comes with breaking changes so operator beware.
It's nice that GitLab actually prevent you from doing that and give you messages that it's unsupported and direct you to their documentation, which describes the supported upgrade paths...
However, at the same time one cannot help but to wonder about why you can't go from version #1 to version #999 in one go. Most of the software at my dayjob (at least the one that i've written) absolutely can do that - since the DB migrations are fully automated, even if i had to create a lot of pushback against other devs going: "Well, it would be easier just to tell the clients who are running this software to just do X manually when going from version Y to Z."
But GitLab's updates are largely automated (the Chef scripts and PostgreSQL migrations etc.), it's just that for some reason they either don't include all of them or require that you visit certain key points throughout the process, which cannot be skipped (e.g. certain milestone releases, as described in their docs).
Of course, i acknowledge that it's extremely hard to sustain backwards compatibility and i've seen numerous projects start out that way and the devs give up on the idea at first sign of difficulty, since it's not like they care much for that and it doesn't always lead to clear value add - it's a nice to have and they won't earn any less of a salary for making some ops' person's life harder down the line.
I also have automated backups with BackupPC, however i expect software to remain reasonably secure and stable without having to update that often - props to GitLab for disclosing the important releases, but i'm migrating over to Gitea for my personal needs as we speak, even if having someone else manage a GitLab install at work is still like having a superpower (with GitLab CI, GitLab Registry etc.).
I actually wrote an article about how really frequent updates cause problems and lots of churn: https://blog.kronis.dev/articles/never-update-anything (though the title is a bit tongue in cheek, as explained by the disclaimer at the top of the article).
I think the version milestones in Gitlab are akin to dependency changes for self-hosted Gitlab. An example is the Gitlab v9 (?) Postgres upgrade to Postgres v11 I think, it was opt-in for a prior version of Gitlab than required at that version milestone. It's difficult to make db migration scripts for Gitlab,as in your example, that may depend on newer Postgres idioms not available in the legacy db version. So you can't simply support gitlab updates from X to Y version due to underlyng dependency constraints...
Thanks for the insightful discourse.
That's just the thing - more software out there should have a clear separation between the files needed to run it (binaries, other libraries), its configuration (either files, environment variables or a mix of both) and the data that's generated by it.
The binaries and libraries can easily have breaking changes and be incompatible with one another (essentially treat them as a blob that fits together, though dynamic linking muddies this). The configuration can also change, though it should be documented and the binaries should output warnings in the logs in such cases (like GitLab actually already does!). The data should have extra care taken to make it compatible between most versions, with at least forwards only migrations available in all other cases (since backwards compatible migrations are just too hard to do in practice).
Alas, i don't install most software on my servers anymore, merely Docker (or Podman, basically any OCI compatible technology) containers with specific volumes or bind mounts for the persistent data. GitLab is pretty good in this regard with its Omnibus install, though there are certainly a few problems with it if you try to do too many updates or have a non-standard configuration.
I actually wrote more about it and why i just migrated away from GitLab to Gitea, Sonatype Nexus and Drone CI on my blog: https://blog.kronis.dev/articles/goodbye-gitlab-hello-gitea-...
Of course, i'll still use GitLab in my company because there it's someone else's job to keep it running with a hopefully appropriate amount of resources to keep it that way with minimal downtime and all the relevant updates. But at the same time, for certain circumstances (like my memory constrained homelab setup), it makes sense to look into multiple lightweight integrated solutions.
You can actually find more information about what broke for me while doing updates in particular, seemingly something cgroups related with gitaly related stuff not having the write permissions needed inside of the container, which later lead to the embedded PostgreSQL failing catastrophically. In comparison, right now i just have Gitea for similar goals which is a single binary that uses an SQLite database, as well as the other aforementioned tools for CI and storage of artefacts, which are similarly decoupled.
It's probably all about constraints, drawbacks and finding what works for you best!
Thanks for your feedback, agreed. I've created an issue https://gitlab.com/gitlab-org/gitlab/-/issues/353862 - please add additional thoughts and suggestions there as well. Thanks!
8.11.Z -> 8.12.0 -> 8.17.7 -> 9.5.10 -> 10.8.7 -> 11.11.8 -> 12.0.12 -> 12.1.17 -> 12.10.14 -> 13.0.14 -> 13.1.11 -> 13.8.8 -> 13.12.15 -> 14.0.12 -> latest 14.Y.Z
Of course, it's probably not viable due to the significant changes that the actual application undergoes and therefore one just needs to bite the bullet and deal with either constant small updates or few and longer updates for private instances.
But hey, thanks for creating the issue and best of luck!
The fact that any non-protected runner can push to it makes it useless to store images to be used in other CI pipeline (unless I missed something, which I would be really glad)
> any non-protected runner can push to it
My understanding is the job token inherits its permission set from the user causing the job to run. If the user has `write_registry` to a project (developer up), then the job does. Do you see more access than that?
The access can be limited per project to specific projects by setting a scope [0] but your description sounds like it might access within the project that is the issue.
0: https://gitlab.sam3.io/help/ci/jobs/ci_job_token#configure-t...
Having a separate production registry is good indeed, but for images to be used for CI itself, having something self-contained within gitlab would have been nice.
why would you want it to be a public instance?