Hacker News Reader

Comment by Corrado | Hacker News Reader

I guess I'm confused as to how the support of one compiled language differs from any other compiled language; couldn't you just replace "Go" with "Rust" at this point? I understand how Lambda would have to support scripting languages (NodeJS, Python, etc.) with built-in runtimes and such, but if you are going to compile down to a binary object anyway, why does it matter what language the binary object was written in?

Does it have to do with supporting libraries being pre-installed on the Lambda image or something? Couldn't they build a Lambda image that handles any language that can interface to common "C" headers?

View on HN

The answer lies in understanding how lambda truly works. Lambda does not start a new version of your program every time the function is called. The real entry point to the program is somewhere else hidden to customers. That program starts up and then waits for requests like a web server. On every request, it invokes our lambda function. It literally invokes the function, it does not spin up a new instance of the program. The program lambda spawns keeps running until no new requests come in for a certain amount of time which is not known. This is also why first invocation of a lambda function after a long time is slower.

Also, there still are other things to take care of. Online code editor support, making sure everything works and there are no bugs, support for compiling the programs edited directly from the browser, etc etc. Also, Lambda support means that the language has access to it's execution context through an API and there is no better way than to implement it in the same language and just pass an object to your handler.

Lambda functions in reality are not one-off CLI commands that boot and terminate fast. They actually spin up a long running service and then the same instance handles multiple requests until the instance is killed off because of being inactive for too long. Having lambda support means AWS implementing a long running server that integrates with your code, handles things like errors, exceptions, logging and provides you with an API to execution context.

Thanks for explanation, it's somehow clearer to me, now. There's one thing I still don't understand: doesn't this mean it's basically just a small server app with a standardized api (as in, api must be implemented in a certain way which allow to call its endpoints as if we were calling functions directly)? What is the specificity of lambda? (beside the pricing style, making pay for requests rather than app execution time)

Yes, in a way. You can also use it to run code on data in S3 or other aws services.

For example I need to munge some data somewhat regularly, which is stored as json on S3. I can trigger it, and have about a thousand processes running in a few seconds, and my large chunk of json processed very quickly. I don't need to maintain any systems for this to work and it's cheap enough it's not worth me worrying about.

Though S3 select might supercede a bunch of my use cases.

Oh, I see: it's not just that it's a simple app with payment on endpoint access, it's that it can spawn tons on this app very quickly.

Am I correct to assume to ideal use case for that is to replace what is typically implemented as a background workers for webservices?

I've not used it much outside my own area but yes I think that's a very good use case. Particularly if there's a "process some files" part. The classic example is thumbnails for images, you can point a very simple bit of imagemagik calling code at a bucket and say "any time a file is dropped in here, create a thumbnail and put it over here" and it'll scale when you have a bunch of images dropped in and cost nothing when nothing is happening.

Not perfect for all use cases, although some people do seem to enjoy running as much stuff in lambda as possible, but when it matches what you need it can be a surprisingly simple solution to quite annoying problems.

Yes, you are right. That's almost it. There are even some open source function-as-a-service platforms out there that do this. The advantage with lambda is that it is dealt with by Amazon so it is virtually infinitely scalable. Also, it integrates with tightly with a ton of AWS services which makes it a great option if you are already invested in AWS.

I see, thanks for confirming this. I was initially skeptic about lambda when it was first released, thinking "so after heroku which makes us pay per process, now we must pay per function call?", but I know better than trusting my doubts about a new tech, I've seen too many people focusing on the parts from a new tech resembling an old one, totally missing what is new.

Would you agree with this statement? => what sets lambda apart is that it boots so fast that it's legit to start one only to call one function, which in turns, thanks to aws infrastructure, allows to massively parallelize short lived tasks.

Yes, I'd agree with that but to me Lambda is much more than that. It's not only suitable for very short running tasks. In fact it is suitable for everything other than very long running tasks. We can build proper APIs and web services on top of lambda and literally only pay for the exact resources we consume. We never have to worry about managing infrastructure, handling auto-scaling, etc. AWS recently also introduced a "serverless" database offering that also comes up when you need it and then goes to sleep. It can scale "infinitely" as well.

I see majority of my workloads implemented on top of Lambda + ECS/Fargate/Kubernestes in near future. Lambda for all short running jobs (1-5mins) like image processing, emails, notifications, web services, APIs, glue code, state machines, etc and ECS for long running tasks like data processing, DB syncs, backups, video encoding etc etc.

> Thanks for explanation, it's somehow clearer to me, now.

Seems weird to completely disarm your "thanks" with "somehow".

He probably meant somewhat.

English is hard.

Indeed, TIL they are not synonymous :) (non native speaker)

So... anything that can be contorted into using Go's calling convention for that one API entry point and embedding something that looks like a symbol table could do the thing?

Probably. That's exactly what some folks did to use the Python runtime to run Go before it was officially supported: https://github.com/eawsy/aws-lambda-go

You can actually sort of do what you are asking. It was common before official support for languages to use node, and simply call out to a binary compiled for Amazon Linux. That does work. However, the support means that the Lambda system can call the specific binary directly, and that libraries are available to expose the data structures easily. There is nothing preventing you from calling a rust program from the node runtime for instance, if you wanted to.

Your code is wrapped so you do not have to deal with networking and serialization. Your code must only satisfy the interface and feels native to the language of choice.

Rust support could be good, but does that mean a lot of work?

I mean, a lots of features that natively supported in Go require crates or coding-by-hand in Rust, maybe that's why Go gets supported first.

The reason is that Go is more mainstream.

And Go has supported cross compilation from the very beginning. Cross compiling in Rust has not been historically that good or accessible for beginners.

Xargo isn't that bad, and neither is using a Docker container to build. Plus, a lot of teams generate their build artifacts in CI environments that are already Linux. So there's a bunch of pretty easy ways to get Rust code running in Lambda.

Honestly the reason why official Rust support may never make sense is that it's so easy to use Neon to build native Node modules with Rust. Not one line of JS is needed and the impedance between the two languages is minimal. Go would require more hacks and copying thanks to both it and Node having their own GC. Basically Rust is flexible enough that official support isn't necessary whereas Go requires official support to avoid ugly hacks.

FWIW, I've never used Lambda to do what I'm suggesting above, but I've done it with GCP's equivalent and it works great.

> why official Rust support may never make sense is that it's so easy

I suspect the only thing Amazon will care about here is consumer demand, and potentially the ability of consumers to move from other platforms to theirs (e.g. if Google cloud supports <thing> they are more likely to do so in order to convince people to move).

Rust is a cool language, but it's use in HTTP services is not as widespread as Go, and there are a bunch of other languages I'd expect first (PHP, Ruby?)

I would assume that the Lambda team is more likely to fill in gaps in support for things that already have official SDKs before starting to officially support something that the rest of AWS isn't yet working on.

Reply on news.ycombinator.com