Hacker News Reader

Just to add more to the post -- there are lots of options outside of SSH-to-the-box and kubernetes out there, you don't have to go to the new hotness quite so quick. I heavily considered:

Dokku (https://github.com/dokku/dokku) Flynn (https://flynn.io/)

I didn't discuss them in the blog post I linked everywhere (I should have), but they're a great option for no-worries deployment, and simpler to set up than kubernetes. I didn't ultimately use them because I felt even messing with copying code to a server wasn't the way forward -- I wanted to just pass a container, and be done, not even adhere to some heroku buildpack (even though they're pretty darn convenient) or anything, just push a container up and things are in production.

OpenFaaS is an "open platform" - supporting Docker Swarm, Kubernetes and whatever else you would like to write a back-end for.

Kubernetes jobs are a potential back-end for asynchronous processing of functions (only). Right now we're achieving that through a NATS Streaming integration - which is agnostic of the orchestration platform.

Checkout the video walk-through for OpenFaaS on K8s.

> Personal Anecdote: I switched all my personal project infrastructure to use Kubernetes (granted, it's a single node cluster), and went through the process of setting it up myself (no tectonic). The process was difficult but immensely rewarding (I have a super long blog post series incoming about it), and various bits of documentation helped me every step of the way. For the curious, my latest ops iteration was docker-containers-managed-by-systemd-provisioned-by-ansible setup to docker-containers-managed-by-kubernetes-provisioned-by-ansible-on-coreos.

I’ve done something very similar myself, and you’re exactly right.

The effort to set up a new project, or tear it down, is basically zero. I can properly version my entire system via git, by just exporting the configs as json, putting them in a git repo, and importing them back if I want to.

I honestly recommend everyone to at least try it out, it’s immensely useful (and a good skill to have)

Yup -- whole heartedly agree. It's so easy that at this point, most of my worry is wondering how I would deal with a disaster scenario, whether the server getting wiped or any other kind of catastrophic failure -- is my automation good enough that I can go from bare-machine to all my projects fully running?

One thing I was a little disappointed with was the ignition provisioning system (https://coreos.com/ignition/docs/latest/) -- it's revolutionary (I mean not really, but it is to me) in the amount of the setting-up-the-server routine that you can skip, but I was disappointed that you can only run it once (by design, that's exactly what it's supposed to do, set up the machine ONCE). I currently manage one larger dedicated host that is more pet than cattle, but if I was truly in someone's cloud spinning up instances, then I could take time to build my perfect ignition config, but I only got one shot (and it worked the first time so), and didn't want to wipe the harddrive.

I do this today with Terraform. If you have any existing cluster, disaster recovery is as easy as swapping the name in your tfvars, then running a new apply. [1] All new infra + kubernetes is set up. I then have a very simple bash script that is just ~30 `kubectl create` lines. I executed this when our automatic clean up script nuked my cluster due to incorrect tagging on my part. Time from monitoring critical alert to back up was under an hour.

[1]: https://coreos.com/tectonic/docs/latest/install/aws/manual-b...

The link you posted is to AWS -- have you ever used terraform on a bare-metal machine? I feel like I've said this at least 5 times so far, but every time I read the Terraform documentation, I get the distinct feeling it's not really for "bare metal" (that's what everyone seems to be calling a machine you actually own in someone's data center)

I really want to use Terraform, but it just doesn't look like it really wants to do the single server use case.

> Yup -- whole heartedly agree. It's so easy that at this point, most of my worry is wondering how I would deal with a disaster scenario, whether the server getting wiped or any other kind of catastrophic failure -- is my automation good enough that I can go from bare-machine to all my projects fully running?

I’m running my system currently on Ubuntu (I’d have to pay extra for additional iSCSI drives), and I had to recreate the system hundreds of times due to mistakes.

So I built an automated system to do that, and tested it hundreds of times. So I’m at least sure that mine works.

Would you mind sharing the system or what it looks like?

I haven't tested the re-build process, which is why I'm almost certain it would fail me/require a lot of terrible manual steps.

It’s thousands of shell scripts that test the status, set package sources, install kubernetes, and install the actual services. Very hackily implemented.

Would you mind providing a link to your blog? Definitely interested in reading your post when you publish it.

Sorry I didn't do the usual "link in profile thing", but here's a link to my blog post on the current setup (ansible): https://vadosware.io/post/ansible-is-awesome/. That post in particular is a bit exuberant, but I did really like ansible that much when I started using it -- it's so rare to have a tool that works just at the abstraction level you want and with most things just working.

I'm literally about to head on a vacation which should really give me more time to write, and the kubernetes posts are next (there is one post in between on building a static haskell executable but I might skip it since the kubernetes stuff is so long). I wrote them as I was going through it so they're messy, and tectonic (terraform) or `kubeadm` are the current "blessed" way to do it on most systems but I enjoyed setting it up myself.

Awesome. Thanks for sharing.

Having tried both Kubernetes and Docker Swarm, I have to say Kubernetes has a much steeper learning curve but is a much more stable and consistent tool that focuses on its set boundaries.

Mesos is sometimes referred to as an alternative, but in reality, it's more of a lower level OS for the server workload than a container native orchestration utility.

I think using Kubernetes as a backend for any FaaS (and in particular OpenFaaS) is a great idea. Can't wait to try it out.

We use Cloud 66 to setup Kubernetes clusters and deploy our stacks to them: https://www.cloud66.com/containers

I agree that there's a steeper curve with K8s - I have a short blog series on it - http://blog.alexellis.io/tag/k8s/

Could I get a link to that blog of yours?

Sorry about that, here's the post on ansible -- https://vadosware.io/post/ansible-is-awesome/

I switched to that infrastructure (I describe the preceeding Makefile+SSH setup a little bit in the post). The kubernetes posts are pretty much next in the pipe, since people have asked I'm going to probably start translating it this week (I wrote rough notes as I went, there are a LOT of notes).

That's exactly how I set up my own cluster running outside of well integrated and isolated platforms (AWS, GCE, you name it). The experiences gathered during this process are available in the shape of a guide, including Terraform modules for fully automated provisioning:

https://github.com/hobby-kube/guide

I would very much appreciate a link to your blog as well

Here's the post on the ansible setup -- it's mostly ramblings but hope it's worth a read, I think the most interesting part is the part where I break down what I think the "levels" of devops are (what I call "operational intelligence", in an attempt to make a new garbage-tier buzzword): https://vadosware.io/post/ansible-is-awesome/

I'm in the process of doing this with Rancher -- is it worthwhile to use both to see the difference, or go with Kubernetes?

Hey so I actually went through that exact same decision. So "Rancher" means one of at least two things (I'm not an expert but just tried really hard to RTFM) -- RancherOS or Rancher the container platform management system -- for those who may not be familiar.

As another person noted Rancher sits at a level of abstraction ABOVE kubernetes -- you can set up Rancher to work with Docker Swarm, Mesos, Kubernetes, and Rancher's own libraries. Since this was how I understood it, I went with just vanilla Kubernetes first, and figured that once I got really comfortable with that, I could throw the Rancher UI on top of it (the install documentation is actually super duper short, it's just another container) when I got tired of typing things in at the console.

I didn't opt to go with RancherOS because CoreOS was already a step away from what I knew -- most of my machines are either arch, alpine (inside a container) or debian/ubuntu. RancherOS takes the stripped-down OS paradigm even further by dockerizing a bunch of system services -- I wasn't quite ready for that. CoreOS not coming with a package manager is already jarring enough for me (I'm used to it a little bit more now) -- I wanted to take a small step rather than a huge one.

Rancher actually uses Kubernetes, they're slightly different tools.

CAN use Kubernetes.

They support Cattle (Rancher's own), Kubernetes, or Swarm for orchestration.

I talked with some of the folks that work at Rancher at the last Dockercon and IIRC they even had some rudimentary Mesos support.

Process of doing which part? An OpenFaaS backend?