Comment by roblabla | Hacker News Reader

Comment by roblabla | Hacker News Reader

Parent Full thread

I just updated the NixOS gitlab package from 8 to 9[0]. It was a nightmarish experience. There are 5 microservices :

- gitlab (the core)

- gitlab-sidekiq (a work queue)

- gitlab-workhorse (provides the frontend)

- gitaly (a git wrapper that caches stuff)

- gitlab-shell (a shell spawned when doing your git clone)

Those are written in either go or ruby. Sometimes mixing the two in the same repository. In the main gitlab repo, there is also some unvendored js dependencies for the frontend, necessitating to jump through some more hoops.

Gitlab has a bunch of hardcoded paths to logfiles and config files. Some config files are toml, others are yaml. Different services need different configs, sometimes duplicating the config entries in a different format.

I love gitlab as a product. But as a sysadmin, it's one of the worst thing I've ever had to deploy.

[0]: Well, it's not pulled there, but it's at https://github.com/NixOS/nixpkgs/pull/27159 if you're interested.

Thanks for the feedback. I'm on the build team here at GitLab, and we try and keep installation as easy as we can, but GitLab is a complicated application.

One of the benefits of the omnibus package, is that all configuration is done through a central gitlab.rb file. So the different configuration formats for the individual services is not noticeable to the administrator. Unfortunately, this doesn't necessarily translate to source installations.

We try and provide good documentation for building from source when our omnibus package or docker image isn't an option: https://docs.gitlab.com/ce/install/installation.html. Any feedback on issues you might have following this guide is welcome. I don't know that we can fix the different configuration formats, but we should be able to fix hardcoded paths. Any examples you could provide would be appreciated.

We do provide docker images for those on operating systems we do not support. https://docs.gitlab.com/omnibus/docker/. If you're just looking to run the latest version, this might be the easiest way for you to proceed.

We have a patch for gitlab in nixos that fixes a lot of our problems. I don't know why we never tried to upstream it, but I guess it's just that nobody bothered. I'll try cleaning it up and upstreaming it when I have some time.

For the omnibus, I'll try looking at how it works, see if it fits our needs better. But I suppose gitaly, which is written in go, still has its own configuration elsewhere, in which you have to replicate the storage list ? That's the sort of thing that I find sort of annoying. It's not a huge issue, but having to manage a bunch of configs, making sure they're all synchronized and up to date sucks as a user experience.

I do have to say, your upgrade guides are very nice and have been helpful when upgrading our instance :).

As far as docker and other things go, I considered it while fighting to get gitlab 9 to work on bare nixos. But the thing is, I manage all my services through nixos, having a central place for all my configuration. This[0] is my configuration (well, a very old one). Everything is in one place, and everything can cross-reference each-other. See my firewall for instance[1], that cross-references various other service's configuration to get their port.

With docker, this is much harder to achieve.

[0]: https://gist.github.com/roblabla/8d1555ceb202eddb1b77

[1]: https://gist.github.com/roblabla/8d1555ceb202eddb1b77#file-c...

The omnibus package is similar to the nixos way of things, except focused on one application rather than the whole system. Once the package is installed, it uses a configuration file, and a set of Chef cookbooks/recipes[0] to configure the individual components.

So yes, Gitaly does come with it's own configuration file that needs to be managed. It does use the same paths as the main gitlab-rails application, so our cookbook has a method to convert the configuration of one to the other[1]

[0]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/fil...

[1]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/fil...

This was my same experience. I spent three solid work days trying to install it from source since it's on a server with other services and their default nginx etc. setup couldn't work. All the different microservices made installing from source very difficult. I ended up finally giving up and using the black box omnibus installation and fiddling with the configuration file for a while.

It works now but I can't hack the source to make custom improvements, so it ends up not being as open source as I would have hoped. There's one particular issue in trying to push that results from being on a URL subpath (http://.../gitlab). I can't fix it myself because of that.

Don't get me wrong--it's a good program and has worked for the team I support. But click and deploy is not useful for people who don't have bare bones servers they can spin up and instead have to work with shared servers.

I gave up and ran a docker container with reverse nginx vhost proxy to the docker port.

Please use the official Omnibus packages to ensure that upgrades are less likely to break.

I think we use toml for the Runner since it is written in Go and should be deployed on another server. The rest is mostly in yml files although we trying to move as much as possible to the UI to make it more user friendly.

> I think we use toml for the Runner since it is written in Go and should be deployed on another server.

Couldn't you just parse YAML in Go, or TOML in Ruby? (Or why not give the option of multiple formats?)

> Please use the official Omnibus packages to ensure that upgrades are less likely to break.

That would be almost precisely against the whole idea of NixOS.

This is the kind of software one expects from a team who has never seen how it's supposed to work in real life.

Reply on news.ycombinator.com