Hacker News Reader

[fission developer here]

About that 100msec overhead -- there's a bunch of optimizations yet to be done, and also we think we can reduce the median at the expense of tail latency; but for now 100msec is good enough performance for a good enough set of use cases (but of course we'd love to hear from people for whom it really doesn't work at all).

Also, we could add knobs that let you tune how long a function stays "warm", so you can tune the latency overhead vs. resource consumption trade-off.

> But with modern progress in fast-starting compiled languages like Go and Rust, I'd love to have some easy way to supply small static binaries instead of source code.

We're very interested in this use case. We're redesigning the function environment stuff right now to better support compiled languages, and allowing users to bypass the source-code level stuff is quite possible.

BTW, to be clear, for a compiled language Fission wouldn't compile source code at runtime; there would be separate build stage that would run when you add the function to fission. I'm also not sure why you say we pay interpreter startup overhead on every request (we don't).

Everything old is new again.

So in the old days there was a way to supply a binary rather than source code. That binary could be fast starting. There was even an open standard for connecting those binaries to different web servers. The standard used unix patterns such as reading stdin and writing stdout.

Sometimes it seems like people have created an insanely high learning curve to run their CGI programs. Sometimes it seems like most of the magic and value add of some of these platforms is that they reduce some of the overhead they added.

Someone needs to start a "container global initiative", and then we can keep all of our binary containers in "cgi-bin".

The cold start impact is quite high, but that's only for APIs which have been idle or are scaling up rapidly.

If you were getting even 1 request per second, the container last used would remain live, so the response time would then be in line with normal responses from a typical NodeJS server running the same code.

I agree though, supplying binaries in something like Go or Rust will be a very nice addition.

They're talking about cold start, not long-term performance. After the initial warm up period, performance should be as fast as Node/Python can execute it. It's the same experience for Lambda and Cloud Functions in my experience.

> But with modern progress in fast-starting compiled languages like Go and Rust, I'd love to have some easy way to supply small static binaries instead of source code.

Isn't that just what "regular" Kubernetes is, except that you wrap a Linux image/container around your static ELF binary? The Stack build tool for Haskell has an option for building as a Docker image, wrapped around the static Linux ELF binary that it would otherwise produce. I assume something similar exists for Go and Rust.

Yep, this is exactly right. The tradeoff here (and it's a good one) is you don't require any binaries at all.

Disclosure: I work at Google on Kubernetes.

I always thought that things like Fission are best for things that will take long enough that 200ms doesn't make much of a difference such as video encoding, updating recommendations, etc. What reducing the overhead does is make it useful for more things like machine vision for images (eg: is this user uploaded image safe for work) but its still nowhere near good enough for doing a DB query that takes 1ms or serving static content.

Warm Lambdas (using Zappa's auto keep-warm) are actually 20-40ms, which makes them suitable for web servers.

In case anyone else was wondering how the auto keep-warm feature is implemented: it's a CloudWatch Event that runs the Lambda every four minutes by default.

While this probably keeps the majority of the requests warm, I would expect that there would still be some cold ones every now and then. Amazon doesn't provide a guarantee in how the containers get recycled.

This is just me speculating, though.

We did some analysis[1] on cold starts in AWS Lambda, and found that there is a cold start if there's no activity for 5 minutes, or regardless of activity, there will be a cold start every 4 hours from creation of the underlying VM.

[1] - https://www.iopipe.com/2016/09/understanding-aws-lambda-cold...

Thanks, I was hoping someone would come along with some real data!