Hacker News Reader

Comment by zalmoxes | Hacker News Reader

I recently(past 6 months) joined a new startup as the operations person, and we standardized on kubernetes for deployment. In the past I've worked with puppet/chef/ansible/heroku/aws/appengine/vmware you name it, and Kubernetes is the nicest and most flexible platform to build on top.

There's a learning curve, and new features are being added, but at this point I would not hesitate to recommend Kubernetes to just about anyone.

CI: We standardize on CircleCI and it gets the job done, but has some serious shortcomings. I've also come close to building my own on top of the k8s cluster and it's not the correct time investment for me right now, but I'd consider building my own in the future. I've yet to find a CI framework I really like.

View on HN

Have you looked at buildkite (http://buildkite.com)? It's a very different approach to CI - you use your own machines. That means you have access to your own s3 buckets, you can talk to your other services and get to use all the CPU and ram your machine has.

We switched over from Circle and it's been awesome to scale our build process without fear of resource constraints. It's pretty great :). They even have a CF setup that will auto scale boxes based on CI load.

We also use Buildkite. Absolutely love it. Support and troubleshooting is very easy to come by as well :)

Well that's exciting. I've been pushing towards Kubernetes with the company I'm at right now, got half the battle done by dockerizing our services and using Harbor to manage the containers.

Do you have any advice, or gotchas on getting into kubernetes development?

Have you had a proper look at Gitlab CI? It know, it's hard to make that sound credible with all that's going on with Gitlab right now but I found their CI to be the best around and you can even use your own machines to run stuff on.

Yes, out of all solutions I'm most excited about GitlabCI and Drone.io

I've worked with a self hosted Gitlab instance before and that was nice, but I'm not quite sure how well the Gitlab CI would integrate when the code is on github.

To be fair, GitLab's current issues are purely operational on the GitLab.com side: their backup strategy and testing needs some work, but the product is really solid and most of the features are already there in GitLab CE.

For me, getting GitLab up and running on kubernetes was a breeze (using the popular docker image [1]) and my `pg_dump; duplicity;` backups are chugging right along. I haven't played with their CI yet, but I'm pretty excited to see how much it can do for me automatically managing my cluster.

[1] https://github.com/sameersbn/docker-gitlab

If you are doing a lot with docker images, it might be worth considering Concourse CI (https://concourse.ci). I haven't played with it in any depth yet so I'm not sure how it compares with other solutions, but I like the idea of your entire build pipeline running through docker images. Theoretically you should even be able to run it in kubernetes.

There's a github repo where someone mounted Concourse on kubernetes[0], but it looks a bit stale. Given the evolutionary pace of k8s, YMMV.

[0] https://github.com/vyshane/concourse-kubernetes

unfortunately concourse when not deployed with bosh does not have any automation (spawning workers, auto-discovery).

Any thoughts on Concourse?

I like it- strong, simple building blocks. It reminds me of Ansible, in that the design seems well factored.

I like Concourse from a manageability perspective, as compared to Jenkins, in that it is more of a "bring your own plugin" in a "just in time" fashion via resource-types-as-docker-images as opposed to requiring a priori provisioning/support of an installation-wide plugin/suite.

It is also very nice in that you can execute pipelines, and even just jobs, on the Concourse installation from your local CLI. This makes it much simpler to test pipeline/job tweaks without a single commit being necessary until it is verified to work. Yes, you can do something similar with the Jenkins Pipeline plugin suite using the Replay functionality but it is significantly more tedious (cut and paste from editor into a textbox in the Jenkins UI, bleh).

Take a look at http://runnable.com

What shortcomings did you find in Circle?

Reply on news.ycombinator.com