Hacker News Reader

Comment by ratiolat | Hacker News Reader

I'm trying to set up a deployment pipeline, quite from scratch. That also includes picking source repository software. Self-hosted is a must for every component. So I chose gitlab and after that I dived into it. Oh boy was I in for a surprise. For CI/CD one needs docker, kubernetes and whatnot: https://docs.gitlab.com/ee/topics/autodevops/ But the alternatives don't look better either.

So, fellow hn'ians. My requirements ATM are quite simple: Submit code via git, run automated test, if all good, put the files into the live system. Pull-requests are must and everything must be self-hosted because cloud is just another persons computer. Code is written in python. What would you recommend? Edit: grammar.

View on HN

I don’t mean this snarky at all, but I’m stunned by the fact that you think a distributed container orchestration platform (which Kubernetes is) is needed to do CI/CD, even when self hosted. It shows I guess how hazy these topics still are. Have a look at Jenkins or Gitlab or Bamboo or any of the other dozen of well known CI server out there. They will pull, test and build your code. Then look at something like dokku or Capistrano or even Ansible to take care of putting builds on servers. With a little bash scripting you have a full pipeline. No Kubernetes needed at all.

Believe me I don't wan't to dive into Kubernetes and learn a beast like that for CI/CD but it says so in the GitLab documentation referenced in my original comment.

The documentation link you shared was for AutoDevOps, a different (but related) feature from CI/CD. It's a build and deployment solution specifically for Kubernetes/Containers, which is why it has those requirements.

GitLab CI/CD documentation can be found at the link that I shared, and it can build and deploy truly all kinds of software, and does not require k8s.

Hey there, I'm the product manager for CI/CD at GitLab. AutoDevOps is one of our more advanced features to get up and running in a k8s environment quickly, but is not required if you just want basic CI.

Our documentation on getting started with CI/CD is here: https://docs.gitlab.com/ee/ci/

I might be wrong about this, so feel free to correct me. Since you're here...

AutoDevOps has a good example of GitLab licensing that frustrates me. It's the "Support for multiple Kubernetes clusters" feature. Running development and production in separate environments seems like a best practice. A mistake that brings down a cluster is really the type of thing you'd want to catch in development or testing, right? However, IMO, if you're a small developer using Core or Starter, GitLab is forcing you to do something subpar by using the same cluster for development and production.

I think "Environment-specific variables" is the same situation where Core and Starter users are lead into bad practices. For example, ASP.NET Core uses ASPNETCORE_ENVIRONMENT which (unfortunately) defaults to "production". The lack of environment specific varaibles means people will put that in the job config of .gitlab-ci.yml.

  deploy_dev:
    variables:
      ASPNETCORE_ENVIRONMENT: "Development"

It also means protected pipelines will be given all the variables needed for deploying to all environments. Ex: DEVELOPMENT_DB,PRODUCTION_DB. I'm sure you can see where this is going, but, if you're using something like EF Core migrations, all it takes is for someone to mess up the (ex:) "deploy_dev" job's ASPNETCORE_ENVIRONMENT variable on protected branch and the CI build is going to end up in "production" mode with access to all the variables needed to break a production DB.

GitLab Flow has a diagram showing master, which is a protected branch, deploying to staging with another branch used for production. I like that workflow, but it means all deployable branches end up being protected branches (I think that's the intent ?) which means production credentials end up exposed everywhere but feature branches.

I don't know how protected variables interact with environment specific variables, but, ideally, I'd want to have ASPNETCORE_ENVIRONMENT be an environment specific variable and PRODUCTION_DB be both environment specific AND protected. The more safeguards the better and navigating into the "production" environment variables page is a good way for me to get into "be super careful" mode.

I think small developers will tend to be the ones where people merge their own changes, so they're especially vulnerable to the type of mistake I just described.

Thanks for the feedback Don.

For your particular use case, you could use the same cluster for multiple environments if you have separate namespaces and use separate service accounts for each. Using RBAC will further ensure you don't have any collisions, this way if there's an error in development, the production namespace will remain untouched.

When deciding which features are paid vs free we ask ourselves if the particular feature may be more relevant to larger organizations; that doesn't necessarily mean there won't be a use case at smaller companies, but just wanted to let you know our reasoning. You can read more about it here: https://about.gitlab.com/stewardship#what-features-are-paid-....

I understand it's tough to tier the features and I almost wouldn't have a problem with all the K8Ns stuff being in a higher tier. I'm not really the target market for it. I just want to learn about it because I think it could trickle down to smaller developers in a few years.

The environment specific variables are more frustrating for me because I would consider good environment isolation a necessity, rather than something that's subjectively better / worse depending on the size of an organization.

I think everyone driving deployment from CI should, at a minimum, start with two environments - 'dev' and 'prod'. The number of environments would have been a much better feature to tier IMO. I don't really need 'test' or 'staging'. To me, those extra environments seem more practical for a company that has a strategy for promoting deployments as a part of their QC policies.

CI in particular is an area where I think the the product tiers might be a mistake. The goal should be getting everyone into the mindset of "use GitLab and follow their opinion for CI and you can one-click deploy to and scale on X, Y, or Z". The promise of cheap, consistent deployment options with the ability to "throw money at it" to scale up is what makes sense for me. It's unlikely I'll ever have a product that gets popular enough that I need to use the scaling features, but that's what everyone dreams about so it's easy to capture the low end with the promise of low / no upfront costs and the ability to scale if you win the lotto.

Thank You! I will look into that. OTOH this is how I came to conclusion that I must go the AutoDevOps way: This is my first installation of gitlab ever. So naturally, after somehow successfully installing it on the 2nd try, everything seems to work. So - lets explore configuration options. For example we should disable registration since, well, we don't want to be a public gitlab host. After a while, after the Pages section, we come to a section named "Continuous Integration and Deployment". And there's a checkbox named "Default to Auto DevOps pipeline for all projects". And there does not seem to be any other kind of "Continuous Integration and Deployment" available so one naturally comes to a conclusion that the only way seems to be Auto DevOps. My bad.

Hi ratiolat, another GitLab PM here. Thanks for the feedback, as jl-gitlab mentioned, we'll work to make that clearer. You can think of Auto DevOps as a CI template that has a job for all devops stages built-in. If you want to experiment with kubernetes first you can easily add a kubernetes cluster from GitLab and experiment using with CI with or without auto devops. More info here: https://docs.gitlab.com/ee/user/project/clusters/

No worries, I understand how you could come to that conclusion. I've opened up an issue to see if we can make that more clear: https://gitlab.com/gitlab-org/gitlab-ce/issues/52144

there are _many_ self hosted alternatives to gitlab. Nothing against them, just saying :) https://gogs.io/ https://gitea.io/en-us/ <= The golang ones are deployed with: Copy the binary, run the binary.

Sounds like you can get away with a bash script that calls rsync, and restarts the code on the server with ssh. (I deploy one of my websites this way, simple & reliable)

If you're looking for something more legit: Ansible

Heroku. It is everything you want.

I'm quite happy with gitea + drone currently.

app engine + cloudbuild.yaml?

Reply on news.ycombinator.com