Hacker News Reader

Comment by seniorsassycat | Hacker News Reader

I don't understand why AWS released Go support instead of binary support and I don't understand why they chose to rely on go's net/rpc.

The Go 1.x runtime will happily execute any binary, go, rust, c++. Events are sent over a TCP stream. That stream uses Go's net/rpc which encodes objects using Go's special [gobs] binary format.

If they had chosen a more common exchange format it would be much easier to support new runtimes. I haven't found any implementation of gobs outside of go. If you write one for a language you could use that language to write a native Lambdas.

[gobs]: https://golang.org/pkg/encoding/gob/

View on HN

part of the "magic" of Lambda is making it easy to use from your programming language of choice. If they made a lower-level model, like, for example, "we start up your binary, and it needs to bind to port X, then we'll send HTTP requests to it and it should respond appropriately using the following JSON specification," it would take a lot more work to set up your Lambda function and get it working correctly than it does now. It's certainly possible for them to do it, but I don't think it fits in with their style.

Worst case, you can implement this interface yourself now easier than ever. Since golang is statically linked and has super fast startup times, you can build an interface where Lambda launches your golang process, and that process binds to a port and spawns a sub-process that implements the actual Lambda function. The golang process would implement the "socket" based interface for you.

People have been doing the above already, but all the existing runtimes, including Node.js, Java, Python, and .NET, have non-trivial startup times and memory usage compared to a golang binary, which should be able to start up in 10s of milliseconds and use less than 1 MB of memory while acting as a proxy.

...or they could support FastCGI or CGI which would then have the "low barrier to entry" and "broad support" already handled..

I think there's a lot of secret sauce involved in order to make Lambdas perform well. They are definitely not just executing a binary every time a request comes in. You might give them a binary but that's not how it's executed.

that's the difference between CGI and FastCGI, though.

I agree. FastCGI or SCGI seem like they would be great solutions, instead of custom rpc formats.

> If they made a lower-level model, like, for example, "we start up your binary, and it needs to bind to port X, then we'll send HTTP requests to it and it should respond appropriately using the following JSON specification," it would take a lot more work to set up your Lambda function and get it working correctly than it does now.

Man, the JSON specification must be annoying/complicated - because the first part of what you said, binding to port X and responding to HTTP, is like.. 1 line in Go. So the first part of what you said is extremely trivial, so the difficulty must lie in the second part?

I wonder what makes the JSON specification so difficult? From your wording, I assume the HTTP response is being sent to AWS, not the actual HTTP requester. Regardless, it must be damn clumsy to justify calling a simple HTTP response too complicated for developers.

Clearly I've never used Lambda though.

I'm only saying that it would take more effort and place more burden on a developer to figure out how to do all that in their programming language rather than just implement a function. I'm surprised this is a contested point.

Tell me in Java, without looking up any online resource, how to set up a FastCGI web application. Can you? Probably not. Now tell me how to write a static method in Java. If you have at least the equivalent Java experience of a freshman undergrad, you can do the latter but not the former.

If you think Lambda should use FastCGI as its common interface, then by all means create a project on github that deploys a Go-based FastCGI bridge to Lambda and makes it easy for you to bundle up your FastCGI application into a Lambda function. Problem solved! Now you just have to get people to use it.

I'm pretty sure anyone who programs Java for a living can start up Jetty via their favourite framework. It isn't as if lambdas are trivial to package either.

> I'm only saying that it would take more effort and place more burden on a developer to figure out how to do all that in their programming language rather than just implement a function

But the average developer isn't going to do that, and won't need to. Within a week there would be more than one framework up on github to do the binding in their favourite language for them. Look at how many web frameworks there are.

> there would be more than one framework up on github

That's annoying, and something I hate about Javascript. There's no standard way to do it, so everyone has a different way of doing it.

I prefer that Lambda has sane, solid standard ways of doing things so if I have to move between projects I know what the heck is going on, rather than having to figure out which of the many homegrown ways of doing things people decided to use.

I wasn't actually thinking about JavaScript at all. It applies to all languages that I have looked at.

I cant see that being an issue here libraries for unofficial languages / runtimes would likely follow the same api as the official ones. There's precedent in the form of unofficial AWS SDKs...

Implementing a Go lambda is already more complicated than writing a function. It requires binding to a TCP port and receiving gob encoded values. AWS released a library that abstracts that into

    // main.go                                                                  
    package main                                                            

    import (                                                                
      "github.com/aws/aws-lambda-go/lambda"                                 
    )                                                                       

    func hello() (string, error) {                                          
      return "Hello ƛ!", nil                                                
    }                                                                       

    func main() {                                                           
      // Make the handler available for Remote Procedure Call by AWS Lambda 
      lambda.Start(hello)                                                   
    }

lambda.Start is the abstraction wrapper.

Why is there a crossbar in the ƛ? Shouldn't it rather be λ?

I think they have to divide by 2 pi to properly normalize it for a distributed filesystem, analogous to quantum's h-bar.

[Edit: this is a joke. I think the person to whom I'm replying has a valid point.]

I'm not sure, I copied that from the aws/aws-lambda-go repo https://github.com/aws/aws-lambda-go

"Parse JSON from a request that arrived over HTTP" is not a major task in any language that's seen serious use since 2002. And if they had done this they would have support for a dozen better languages literally overnight.

The industry has way too many frameworks that should have been protocols.

I’m curious which dozen languages you find better than the ones supported by Lambda.

The point is that no one would have to be debating which languages should be supported by Lambda if they made their environment language-agnostic based on a protocol rather than a bunch of individual libraries.

.NET Core 2.0, until _very_ recently.

Common Lisp. Until just now, Go. I hear great things about Java 9. TCL. Rebol. Smalltalk.

That's half a dozen (-ish); I'll let someone else complete it.

TCL, Smalltalk, Rebol??? So they could support all 5 users of these languages? Lol

They've already implemented a lower-level protocol "we fork/exec your binary, it needs to bind to port _LAMBDA_SERVER_PORT, and it must follow gos net/rpc protocol". aws provided a library, aws-lambda-go, that every go implementer must use.

Anyone can implement their own 'aws-lambda-go' library in what ever language they want, but the choice to use net/rpc and it's gobs format makes it more difficult than it could be. json is implemented in practically every language. there are binary formats with libraries is most languages if your want to be efficient.

Choosing a protocol other than net/rpc would 1. make it easier for AWS to support more runtimes 2. make it easier for the community to support custom runtimes. 3. be as easy to integrate with as aws-lambda-go is.

I don't understand why it can't all be done over standard in / standard out? Why bother with ports at all?

Yup. It used to be called CGI ... In the '90s.

Because then you’d need a (inefficient) fork/execing webserver, or some convoluted (error-prone) way of multiplexing multiple connections over one stream. The OS file descriptor model already handles this for you with listen() and accept().

CGI only works if you’re willing to take the hit of launching an entire process (good luck using any interpreted language, and even go needs some runtime initialization at startup) for every single web request.

Lambda/FaaS is more like FastCGI

https://en.wikipedia.org/wiki/FastCGI

Replying way late here, but FastCGI doesn't work by printing to stdout, it talks to a domain socket or an open TCP port. The grandparent post was asking "why can't lambda just use stdout?"

The magic of TCP or domain sockets is that calling accept() gives you a unique file descriptor that corresponds with a single client.

Just using stdout doesn't work that way, unless your entire process was launched to handle one request, hence CGI being very inefficient.

Because it would make moving away from lambda much easier.

Wait isn't one of the selling points of Go that you can just deploy single statically compiled binaries.

Reply on news.ycombinator.com