Show HN: Kubernetes as a back-end for OpenFaaS | Hacker News Reader

Show HN: Kubernetes as a back-end for OpenFaaS

github.com

194 pointsalexellisuk67 comments

Kubernetes jobs (https://kubernetes.io/docs/concepts/workloads/controllers/jo...) and kubernetes cron jobs (https://kubernetes.io/docs/concepts/workloads/controllers/cr...) make a task like running a fairly robust FaaS infrastructure (this involves things like loadbalancing jobs, tracking state, etc) much simpler -- really excited for a world where people are empowered to create their own clouds rather than get on someone else's.

Personal Anecdote: I switched all my personal project infrastructure to use Kubernetes (granted, it's a single node cluster), and went through the process of setting it up myself (no tectonic). The process was difficult but immensely rewarding (I have a super long blog post series incoming about it), and various bits of documentation helped me every step of the way. For the curious, my latest ops iteration was docker-containers-managed-by-systemd-provisioned-by-ansible setup to docker-containers-managed-by-kubernetes-provisioned-by-ansible-on-coreos.

I say all that to say that after having personal experience both good and bad with kubernetes, I am thoroughly impressed with it, and have become a huge fan. It is internally consistent with it's principles and concepts, and once you have things set up they mostly just work. When things go wrong (as they do inevitably), the concepts are useful tools and generally hold up, and it's often just a case of finding examples, and giving the documentation a more thorough reading.

You can tell when you've learned a truly significant piece of knowledge if you can't stop thinking of all the ways you can apply it. I've had that experience with kubernetes -- just wanted to share, if anyone was on the fence about playing with it.

OpenFaaS is an "open platform" - supporting Docker Swarm, Kubernetes and whatever else you would like to write a back-end for.

Kubernetes jobs are a potential back-end for asynchronous processing of functions (only). Right now we're achieving that through a NATS Streaming integration - which is agnostic of the orchestration platform.

Checkout the video walk-through for OpenFaaS on K8s.

Could I get a link to that blog of yours?

Sorry about that, here's the post on ansible -- https://vadosware.io/post/ansible-is-awesome/

I switched to that infrastructure (I describe the preceeding Makefile+SSH setup a little bit in the post). The kubernetes posts are pretty much next in the pipe, since people have asked I'm going to probably start translating it this week (I wrote rough notes as I went, there are a LOT of notes).

I'm in the process of doing this with Rancher -- is it worthwhile to use both to see the difference, or go with Kubernetes?

Rancher actually uses Kubernetes, they're slightly different tools.

CAN use Kubernetes.

They support Cattle (Rancher's own), Kubernetes, or Swarm for orchestration.

I talked with some of the folks that work at Rancher at the last Dockercon and IIRC they even had some rudimentary Mesos support.

Process of doing which part? An OpenFaaS backend?

Hey so I actually went through that exact same decision. So "Rancher" means one of at least two things (I'm not an expert but just tried really hard to RTFM) -- RancherOS or Rancher the container platform management system -- for those who may not be familiar.

As another person noted Rancher sits at a level of abstraction ABOVE kubernetes -- you can set up Rancher to work with Docker Swarm, Mesos, Kubernetes, and Rancher's own libraries. Since this was how I understood it, I went with just vanilla Kubernetes first, and figured that once I got really comfortable with that, I could throw the Rancher UI on top of it (the install documentation is actually super duper short, it's just another container) when I got tired of typing things in at the console.

I didn't opt to go with RancherOS because CoreOS was already a step away from what I knew -- most of my machines are either arch, alpine (inside a container) or debian/ubuntu. RancherOS takes the stripped-down OS paradigm even further by dockerizing a bunch of system services -- I wasn't quite ready for that. CoreOS not coming with a package manager is already jarring enough for me (I'm used to it a little bit more now) -- I wanted to take a small step rather than a huge one.

> Personal Anecdote: I switched all my personal project infrastructure to use Kubernetes (granted, it's a single node cluster), and went through the process of setting it up myself (no tectonic). The process was difficult but immensely rewarding (I have a super long blog post series incoming about it), and various bits of documentation helped me every step of the way. For the curious, my latest ops iteration was docker-containers-managed-by-systemd-provisioned-by-ansible setup to docker-containers-managed-by-kubernetes-provisioned-by-ansible-on-coreos.

I’ve done something very similar myself, and you’re exactly right.

The effort to set up a new project, or tear it down, is basically zero. I can properly version my entire system via git, by just exporting the configs as json, putting them in a git repo, and importing them back if I want to.

I honestly recommend everyone to at least try it out, it’s immensely useful (and a good skill to have)

Yup -- whole heartedly agree. It's so easy that at this point, most of my worry is wondering how I would deal with a disaster scenario, whether the server getting wiped or any other kind of catastrophic failure -- is my automation good enough that I can go from bare-machine to all my projects fully running?

One thing I was a little disappointed with was the ignition provisioning system (https://coreos.com/ignition/docs/latest/) -- it's revolutionary (I mean not really, but it is to me) in the amount of the setting-up-the-server routine that you can skip, but I was disappointed that you can only run it once (by design, that's exactly what it's supposed to do, set up the machine ONCE). I currently manage one larger dedicated host that is more pet than cattle, but if I was truly in someone's cloud spinning up instances, then I could take time to build my perfect ignition config, but I only got one shot (and it worked the first time so), and didn't want to wipe the harddrive.

> Yup -- whole heartedly agree. It's so easy that at this point, most of my worry is wondering how I would deal with a disaster scenario, whether the server getting wiped or any other kind of catastrophic failure -- is my automation good enough that I can go from bare-machine to all my projects fully running?

I’m running my system currently on Ubuntu (I’d have to pay extra for additional iSCSI drives), and I had to recreate the system hundreds of times due to mistakes.

So I built an automated system to do that, and tested it hundreds of times. So I’m at least sure that mine works.

Would you mind sharing the system or what it looks like?

I haven't tested the re-build process, which is why I'm almost certain it would fail me/require a lot of terrible manual steps.

It’s thousands of shell scripts that test the status, set package sources, install kubernetes, and install the actual services. Very hackily implemented.

I do this today with Terraform. If you have any existing cluster, disaster recovery is as easy as swapping the name in your tfvars, then running a new apply. [1] All new infra + kubernetes is set up. I then have a very simple bash script that is just ~30 `kubectl create` lines. I executed this when our automatic clean up script nuked my cluster due to incorrect tagging on my part. Time from monitoring critical alert to back up was under an hour.

[1]: https://coreos.com/tectonic/docs/latest/install/aws/manual-b...

The link you posted is to AWS -- have you ever used terraform on a bare-metal machine? I feel like I've said this at least 5 times so far, but every time I read the Terraform documentation, I get the distinct feeling it's not really for "bare metal" (that's what everyone seems to be calling a machine you actually own in someone's data center)

I really want to use Terraform, but it just doesn't look like it really wants to do the single server use case.

Would you mind providing a link to your blog? Definitely interested in reading your post when you publish it.

Sorry I didn't do the usual "link in profile thing", but here's a link to my blog post on the current setup (ansible): https://vadosware.io/post/ansible-is-awesome/. That post in particular is a bit exuberant, but I did really like ansible that much when I started using it -- it's so rare to have a tool that works just at the abstraction level you want and with most things just working.

I'm literally about to head on a vacation which should really give me more time to write, and the kubernetes posts are next (there is one post in between on building a static haskell executable but I might skip it since the kubernetes stuff is so long). I wrote them as I was going through it so they're messy, and tectonic (terraform) or `kubeadm` are the current "blessed" way to do it on most systems but I enjoyed setting it up myself.

Awesome. Thanks for sharing.

I would very much appreciate a link to your blog as well

Here's the post on the ansible setup -- it's mostly ramblings but hope it's worth a read, I think the most interesting part is the part where I break down what I think the "levels" of devops are (what I call "operational intelligence", in an attempt to make a new garbage-tier buzzword): https://vadosware.io/post/ansible-is-awesome/

Just to add more to the post -- there are lots of options outside of SSH-to-the-box and kubernetes out there, you don't have to go to the new hotness quite so quick. I heavily considered:

Dokku (https://github.com/dokku/dokku) Flynn (https://flynn.io/)

I didn't discuss them in the blog post I linked everywhere (I should have), but they're a great option for no-worries deployment, and simpler to set up than kubernetes. I didn't ultimately use them because I felt even messing with copying code to a server wasn't the way forward -- I wanted to just pass a container, and be done, not even adhere to some heroku buildpack (even though they're pretty darn convenient) or anything, just push a container up and things are in production.

That's exactly how I set up my own cluster running outside of well integrated and isolated platforms (AWS, GCE, you name it). The experiences gathered during this process are available in the shape of a guide, including Terraform modules for fully automated provisioning:

https://github.com/hobby-kube/guide

Having tried both Kubernetes and Docker Swarm, I have to say Kubernetes has a much steeper learning curve but is a much more stable and consistent tool that focuses on its set boundaries.

Mesos is sometimes referred to as an alternative, but in reality, it's more of a lower level OS for the server workload than a container native orchestration utility.

I think using Kubernetes as a backend for any FaaS (and in particular OpenFaaS) is a great idea. Can't wait to try it out.

We use Cloud 66 to setup Kubernetes clusters and deploy our stacks to them: https://www.cloud66.com/containers

I agree that there's a steeper curve with K8s - I have a short blog series on it - http://blog.alexellis.io/tag/k8s/

  > Function Watchdog
  > 
  > You can make any Docker image into a serverless function by adding
  > the Function Watchdog (a tiny Golang HTTP server)
  > The Function Watchdog is the entrypoint allowing HTTP requests to be 
  > forwarded to the target process via STDIN. The response is sent back to
  > the caller by writing to STDOUT from your application.

Did they actually just reinvent FastCGI ?

I would guess the "handler" is short-lived, in order to maintain agility in updating the "function", so more like CGI than FastCGI?

FaaS/serverless is very similar to CGI. When talking to new adopters, I describe serverless cynically as an incredibly resource-wasteful rediscovery of CGI using VMs instead of processes; I describe it charitably as a very flexible CGI in which you don't have to think nearly as much about your physical hardware (that runs your webservers) or your platform code (webservers and frameworks, e.g. httpd/wsgi, that need to be configured to provide the right CGI containers/layers) in order to scale.

CGI is a very powerful concept whose initial adoption was hindered primarily by a lack of standardization of API interchange formats. It's experiencing a resurgence now, I think, because we've gotten a lot better at API-first design, and also because orchestration layers have gotten a lot better.

I feel similarly about Docker/containers: they're basically the rediscovery of static linking, made more popular than it was initially due to the present-day abundance of disk resources and the presence of really easy-to-use-at-scale deployment platforms.

Note that I'm not equating static linking/containerization with CGI/serverless in quality. CGI was pretty much universally an improvement over its predecessor. Static linking has and had many flaws, as the Docker crowd is discovering as container-based shops shift into "who built this crazy Dockerfile anyway, and why did they hand-select an ancient version of OpenSSL?!?!" maintenance mode, and ops folks get surprised by the disk, memory, and startup time costs of legacy containerized microservices that turn out (long after their creators move on) to not be so "micro" after all.

What's old is new again; the good and the bad.

Unless you are binding to a limited set of runtimes (i.e. Node and Python in the way Kubeless currently does) then you do need to find a common "language" or "protocol" to speak to functions. STDIN/STDOUT appears to work quite well.

Not reinvented, the author is aware. For example, the Watchdog has an option called "cgi_headers", which controls whether the headers should be parsed and sent as environment vars (like in CGI) or passed raw.

My thoughts exactly. Seems super inefficient. Why not take the Heroku model of sleeping dynos?

Spinning up a new process to handle each request is probably costly.

The FaaS platforms of the three big providers are basically undifferentiated in their feature sets. If anything, the one from Microsoft is the best because it supports more languages.

The real differentiator is the ecosystem that it exists within. It is the glue that binds. And in that regard, AWS is the hands down winner. One, because you can do more stuff with your FaaS, and more importantly, it has more triggers than any other platform.

It is the list of things that can trigger your functions that truly make a FaaS useful.

The problem with self-hosted FaaS is the limited set of triggers, because all the other stuff you run doesn't support triggering the FaaS (yet?).

It's a great programming paradigm, but the true value will come when all the other self hosted projects can emit triggers.

AWS supports the most languages. Azure support for most of the listed languages such as python and php is experimental and not production-ready.

At that point it's just a semantics debate. I mean if you include the ability to use shims, then they all support the same languages.

But I'd still rather run C# on Azure than anywhere else.

Kubeless is another open source Kubernetes-native serverless implementation.

http://kubeless.io/

http://www.codechannels.com/video/containercamp/bitnami/kube...

Kubeless is going to suit you if you only need to run Python or Node.js code. OpenFaaS supports any binary / container for Windows or Linux as a serverless function along with some templates for common languages. This blog post raises some concerns around Kubeless and competition - http://www.devoperandi.com/kubernetes-faas-options-part-1/

This is my wet dream since lambda came out. But how about performance? What's the "cold" response latency compared to lambda?

Other frameworks listed here have 350ms start time - OpenFaas is a number of times faster.

It also depends entirely on the disk storage, RAM generation & speed and CPU. Golang and Python are fastest. I am sure it could be further optimized for a given combination.

Tested on a Type 3 Packet Bare Metal host with latest Docker Swarm. Golang sample markdown function - 0.024s / 24ms. The NodeJS example called NodeInfo comes back in around 0.1s / 100ms for a round-trip. A bash built-in (cat) is 0.015s / 15ms. Python is similar to Golang for start time.

Sorry for the perhaps silly question, but is this completely cold start for the full round-trip, or is the container "pre-warmed" or something like that?

id imagine that could be cold-start time so long as required images were already present on whatever host the job/function got run on.

For languages on the JVM, having a warmed-up JVM makes a big difference (not only because of the JVM startup, but also because the JIT compiler can work it's magic).

Lambda optimizes for this and keeps the JVM around. From this example: https://github.com/alexellis/faas/blob/8ec31d9b89430176c834b... and also from the docs, it looks like the JVM is started for each invocation.

When I looked into funktion, it did the same.

Do you have any plans to optimize for this use case?

It looks like most Open-Source FaaS implementations don't even consider the JVM, so it's really great that OpenFaaS does! I'm planning to play around with it during our next techtime either way, but I'm unfortunately afraid it's too limiting if we have to cold-boot a JVM each time.

Join us on Slack and let's do some testing and figure it out. Email alex@openfaas.com for an invite.

The response time for the JVM on modern hardware looked good - similar to Node.js, but last night I tried re-using the forked process in Java and it appears to work very well. That would mean you only pay for the JVM boot once.

If you or any other Java users/experts are willing to help test you're most welcome in the community.

sscarduzio shoot a mail over to alex@openfaas.com and join us on Slack

Fission is also worth a mention: http://fission.io/, I will certainly keep an eye on `faas-netes`

Checkout the beginner tutorial I posted on this story and the CNCF demo / intro video which gives a good tour and overview - https://youtu.be/SwRjPiqpFTk?t=1m8s

Thanks. Here's the github link for fission: https://github.com/fission/fission

This is exciting! Kubernetes is a clear leader, and fits the role of process scheduling for FaaS really well. Excited to see where this goes!

Kick the tyres with this intro - https://github.com/alexellis/faas/blob/master/guide/deployme... and https://blog.alexellis.io/first-faas-python-function/

Would be great if this were packaged as a Helm chart.

Making a Helm chart available is planned, should subscribe to this issue to get a heads up on when its ready to consume.

https://github.com/alexellis/faas-netes/issues/10

I like the idea, but a swarm as app inside k8s? It's like the same technology, right?

It can use either-or, rather than both together

Purely either Swarm or K8s - or if you like write your own back-end for Marathon/Nomad etc.

"functions as a service"? Is it what it sounds like?

Why?

Can you explain what you mean? If you'd like more intro information checkout - https://blog.alexellis.io/introducing-functions-as-a-service...

I still don't see anything that explicitly distinguish it from "left-pad api" [0]

For example:

> A serverless function is a small, discrete and reusable chunk of code that:

> ... > executes in a few seconds ..

Why would you want a function execution to take several seconds?

I must be missing something ..

[0]: http://left-pad.io/

Probably if the function execution time is too small, the overhead isn't worth it? And if it's too large, there's too little granularity? I guess it's the same ideas that guide "ordinary" programming styles, only adjusted to the timings of large networks (milliseconds instead of microseconds).

You've just cherry-picked a generic industry description of FaaS functions.

"Executes within a few seconds as a rule" - otherwise you're not really building an event-driven API, but something else. You can run the code for however long you like and even invoke via async.

Left pad adds whitespace to a string? Don't see the connection

I don't know. I don't think I cherry-picked. The concept sounded ridiculous to me, so I asked for clarification. The article provided for me (which was supposed to introduce the concept to people who haven't heard of it) only seemed to confirm my suspicions.

I _must_ be missing something. I mean I probably will still think the idea is unsound, but at least I want to really understand the thinking behind it better before I solidify my negative impression.

Here's an example of how Netflix makes use of FaaS: https://www.youtube.com/watch?v=hU25CIRPIJo

Ha! "New Abstraction Layer", presented as a good thing, 0:42

You don't know about leftpad? It must be deployed in a highly available serverless environment.

Leftpad on OpenFaaS - https://github.com/faas-and-furious/faas-leftpad

Well, I'm not as convinced as to the 'why' in one's own private cloud (though if you're large enough to have a true separation of concerns, and a dedicated team to administering your cloud, fair enough), but for a public cloud, the ability to fire off a small snippet of code and pay nearly nothing for it, and do very little to make it production worthy is really beneficial.

It allows you to write rules like "every time I do X, execute this bit of code", without having to spin up servers or capacity plan for it; that's left to the cloud provider to do.

It also allows you to build 'Serverless' style applications, by even making your incoming client calls just execute FaaS to figure out how to act and respond.

Altogether the benefits are the application developer doesn't have to worry about the VM security, scaling, etc. They just write their code, and it scales transparently, on VMs managed by the cloud administrator.

For an immediately obvious use case, if you're in a cloud already, you can execute a FaaS as a cron job. Then, you don't have to worry about "where do I execute this" (is it sharing a VM? Does it get its own VM?), with the associated tradeoffs; you just point the cloud's cron scheduler to the function, and you're done.

If the term SAAS wasn't already so taken, I suspect that the generalized term would be 'Scripts As A Service' instead of FAAS (which might slot into your brain differently).

In the same way you might write a number of small utility scripts piping data between each other for various purposes, you can do the same with FAAS. You just also get the benefit of not having to manage infrastructure, crazy amounts of scale, etc.

Why do I get the impression that it resembles Alan Kay's ideas on how the Internet should have looked like, only in a more heavyweight/baroque incarnation?

Reply on news.ycombinator.com