Hacker News Reader

I concur with you on the resource usage of Gitlab. It's a complex system, and takes up a lot of resources. I haven't been able to get it to run in a stable manner with less than 8GB of RAM on a very moderately used setup.

BUT I very much disagree with your general rejection of it's worth. Yes, the feature set is ballooning, and some of them are very half baked.

But I find tremendous value in a integrated solution that offers code hosting, wikis, issue tracking and a tightly integrated CI without having to jump between 4 different systems.

There are a lot of use cases where Gitlab is overkill, but for others it is a really great solution that even comes for free.

Also, as long as the system has enough resources, I've found it to run very reliably and upgrades are painless. (I also agree though that the backup task is a PITA. Really slow and resource hungry).

Well gitea/gogs also have code hosting, wikis, issue tracking, and while CI is not tightly integrated it is fairly easy to integrate. So the value has to come from something else :) GitLab has integrated chat and kanban boards so there is other cool stuff in Gitlab but it's not really any of your listed reasons.

Edit:

I would have to say if I were to choose GitLab the biggest reason would be the UI. The front-end just looks a lot better. If Gogs/gitea could get a sponsor to help clean it up it would be a lot more popular.

What would you recommend as CI solution for gitea?

I'm looking for something that's self hosted, foss and does not require an admin to set it up for every project. GitLab offers this, but I haven't found anything that competes really.

We use a Gitea for source control and Jenkins for CI. There's an existing jenkins-git plugin that makes connecting them very easy.

We switched from GitHub to Gitea last year and love it. We looked at Gitlab but it reminded me too much of phabricator - trying to be all things to all people, bloating all over. I prefer specialized tools that focus on a smaller core of functionality and interoperate easily with the other pieces.

I‘ve been using drone (http://drone.io). I self host it alongside gitea. The documentation us quite good, loads of plugins available and it works flawlessly with gitea.

Pedantic notice: an utter dearth of counterproductive features would seem to be a good thing rather than a problem.

I was confused by that, too. I have a problem with the word ‘dearth’ in that I can often not remember whether it means an abundance or scarcity. So I looked it up, and found that it means scarcity. Perhaps GP makes the same mistake I do and misused dearth to mean abundance?

Yes, it means scarcity and the GP used it to mean abundance.

My mnemonic is that "dearth" looks like "death".

I suspect they meant more like "profusion."

As they say - less is more.

As another systems engineer, I'd much rather have clearly delineated features and how to interconnect them, rather than a cornucopia of disparate and badly documented "features".

I'm thinking of the class of "Does:X, Y, Z" but Z requires unstated dependency of A and C, and C requires you to have a specific database organization type that was asked at the initialization of the system. And there would be no hint that you had to choose the DB type, and how that would have ramifications 10 steps down the line.

Going back, what I'd rather have is more in line with the Unix philosophy. Do something and do it well. Let me connect somethings to other somethings. But Gitlab is a cathedral, and disorganized at that.

I think you're missing the point as well; "dearth" is a lack of. A lack of counter-productive features is a good thing.

I think you've misunderstood who is responding to who. A "dearth of somewhat counterproductive features" is being described as a bad thing in the OP, using an incorrect/nonexistent definition of "dearth" as "there are too many somewhat counterproductive features".

> gitlab is also so large it requires its own chef deployment to competently install it from omnibus, and has no HA roadmap in sight unless you want to break apart its rube-goldberg structure and attempt to HA the individual components of it.

Use the gitlab-ce docker container and save yourself the hassle. Literally takes less than 10min to set up and a version upgrade is nothing more than docker stop, docker rm, docker run <new version> away.

Only thing I haven't solved with this is HA but you can use a central NFS mountpoint with an HA'd setup there, and simply start a new container on another machine with the NFS mount... will require manual intervention, yes, but it's more efficient than trying to break up the dozens of moving parts.

Thanks for posting. HA is a paid feature of GitLab and will likely stay a paid feature.

We're working hard on a cloud native alternative to the docker container that works well on Kubernetes. There is more information on https://gitlab.com/charts/gitlab/ and you can see the activity on https://gitlab.com/charts/gitlab/commits/master Since it is cloud native I assume it will allow autoscaling and it neatly orders every component in its own container.

Today GitLab is using a lot of RAM and the best way to reduce that is to make the application server multithreaded https://gitlab.com/gitlab-org/gitlab-ce/issues/3592 The first priority is to add rubocop rules for https://github.com/covermymeds/rubocop-thread_safety There is a WIP merge request in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1899 Any help is appreciated.

> Today GitLab is using a lot of RAM and the best way to reduce that is to make the application server multithreaded https://gitlab.com/gitlab-org/gitlab-ce/issues/3592

how is multi-threading helping with memory consumption?

Because the Rails application server is currently single threaded it forks multiple processes that each take up about half a gigabyte of RAM. Running fewer processes would cause increased latency for users.

In a multi-threaded program, the threads share memory. Currently the application server is running in multiple processes, each of which contains a full copy of the core application.

NFS will give you redundancy but not high availability. In fact it will probably lower your availability the first time NFS gets hung.

But if manual intervention works for your setup, then you probably don’t really need HA anyway, just redundancy.

We're working towards Gitaly 1.0 that would allow you to run GitLab without NFS. This isn't HA but it would prevent one NFS server from taking down the whole cluster.

    > gitlab [...] also [...] has no HA roadmap in sight unless you
    > want to break apart its rube-goldberg structure and attempt to
    > HA the individual components of it.

We run a very large in-house GitLab installation and the HA is easy because:

1) GitLab stores data on the FS, and you can outsource this problem to e.g. a NetApp filer.

2) The data it doesn't store on NFS it stores in psql or MySQL, which has established HA solutions.

Once you have that set up you can load-balance the GitLab nodes and provision / destroy them at will.

Thanks, you can indeed do HA this way.

When Gitaly 1.0 lands you won't have to mount the NFS volumes on the application servers anymore as everything will go over gRPC.

That just splits up the task of application server & git executor into two, it doesn't get rid of the dependency on a monolithic storage backend.

I understand that this is something gitlab.com is working towards and excited about, presumably because it'll allow for provisioning web workers unrelated to the CPU you're spending on running git, but for us I don't see it being useful.

I hope GitLab eventually gets something like GitHub Spokes[1] which'll allow for provisioning the entire setup on a bunch of dumb boxes with RAID 0.

1. https://githubengineering.com/introducing-dgit/

The advantage is that if one storage nose is offline the rest of the site can still function.

Spokes is more elaborate can can deal with node failure. We plan to rely on cloud storage so we can quickly attach a new VM with Gitaly.

It's hard to imagine that TCO for NetApp + Gitlab is less than Github or some other hosted solution.

> [...] and implement competent things like competent CI [...]

I've got an OSX build box with a bunch of virtualbox VMs so I can build for Gnu/Linux (i386, x86_64, and arm), OSX, and Windows. Single slow box, so one runner at a time. It has to be running OSX because I haven't found any way to run an OSX VM on any other platform.

I want to trigger CI on Gitlab merge requests.

What does $competent_CI expand to?

Take a look at Veertu, it uses macOS’s built in Hypervisor (like Docker for Mac now does) and integrates the VMs into a pretty decent Jenkins plugin. I’ve been using it since early alphas and it’s one of the best I’ve used in a self hosted solution.

I set up some scripts to build virtual environments, including MacOS under KVM: https://github.com/kstenerud/virtual-builders

Specifically: https://github.com/kstenerud/virtual-builders/tree/master/bu...

> It has to be running OSX because I haven't found any way to run an OSX VM on any other platform.

It is fairly easy to get decent OS X virtualization using Debian as base OS and QEMU-KVM for virtualization. You won't have niceties such as GPU acceleration but it will work.

Search for "OSX KVM" on Google, it will take you where you want - unfortunately a direct link would expose me to legal liabilities. Also, be warned that using this method may violate Apple's EULA.

Hackintosh systems are easy as long as you're not looking to update them after everything is working correctly.

They are fragile systems that one doesn't really want to risk production environments on in my opinion.

Generally Hackintoshes are fine for CI machines, since it doesn’t really matter if Wi-Fi or the camera or iMessage works correctly, as it might matter if you were using the machine for personal use. Updates are also generally fine if you wait a bit for other people to test before installing.

As far as I know, there isn’t anything that goes against using OSX in a VM in the EULA — so long as your host runs on Mac hardware. I don’t think you even need to use OSX as the primary OS in the machine.

That’s an admittedly big caveat, but also one of the reasons why virtualizing OSX works in the first place.

Have you figured out a way to bootstrap it without a Mac? I successfully got an image running under Vbox on Fedora, but I had to build the image first on a Mac and then export it as an appliance.

Also, performance was atrocious. It was so bad as to basically make it unusable. 5 second delay before processing mouse and key events.

It's possible to create an OSX VM under KVM. It's a little clunky, you get very bad graphics performances, and you can run in really weird issues (for example, the NIC not going-up properly), but it works, and if you are using it as a CI platform and/or only sshing into it, it's more than usable.

It's really handy when you are testing/adding support for projects (like C/C++ with cmake) on OSX.

I was about to ask "why not Gitlab?". Thanks for answering it in advance.

I had a class last semester that submitted everything through Gitlab and when we tried to run unit tests to test for the correct code, it would hang for 7-8 minutes. Very frustrating. Happened on all the assignments so was not just a one time thing.

I had a similar class that used Gigs instead and never had any performance issues. It didn't feel as polished as GitLab though.

Could you provide more info about this? What do you mean by "it would hang"? Any specific component that was unresponsive?

Was this on a self-hosted instance or on GitLab.com?

We'd love to make sure this doesn't happen again.

Could you describe these specs in money terms? This reads like it's painful to throw 4 cores and 8 GB RAM at the problem -- what is that costing you?

> [...] programmers asking why git has problems with their newly requested LFS and a 40gb video file they decided to store.

The thing is, git, as our industry uses it, is not really a good tool for keeping artifacts, and "video" sounds like an artifact, not a source file. I'd like to see more artifact repositories, especially the ones that you can talk to from command line or from a Python/Ruby/whatever script. I haven't found any such thing, so I wrote my own.

What is your artifact repository called and what are the main features? I'd love to check it out.

As severine found out, it's called GrailBag (https://github.com/dozzie/grailbag). Docs and README are lacking, to put it mildly, so I don't deem it ready for being published, but I don't exactly hide it either.

From the features, GrailBag stores files along with key:value pairs as metadata (not surprising), has a command line client to list/modify/upload/download/delete the artifacts, and has a Python module for doing the same from a more sophisticated script.

Additionally there is an interpreter of a simple language that describes directory tree where artifacts will be deployed (which artifacts to download, how to name the files, what directories and symlinks to create). This way you can write a cron script that deploys whatever you have in the repository, e.g. in a cold-standby server scenario (two servers having the same configuration, but one shut down and only powered on when the first one fails; on the first run of the cron task, the spare server downloads all the missing files, including whatever got stored while the server was shut down).

Maybe https://github.com/dozzie/grailbag

Yeah I didn’t have much luck with LFS but we didn’t really need it so it wasn’t a problem to just disable.

I love gitlab. I have it running for a small team on a Digital Ocean Droplet it was trivial to get running. If we had gone for self-hosting we’d have gone the docker route. Having CI built in was a massive boon when we switched over. My only complaint really is the resource usage but it isn’t causing problems (8GB). Aside from that I love it.

Exactly my feeling, and I just tried it once on an AWS instance or similar. It was the only serious program running on the machine and it dragged down all the resources, worse than Jenkins ever could. All that doesn't say gittea is much better, though.