Hacker News Reader

Comment by st3fan | Hacker News Reader

One thing that is not super clear from the blog post, and which may be a surpsise to some, is that Cloud Functions will require you to upload your source code. The compilation happens not on your end but on Google servers.

This is probably not a surprise for folks who have been using Python or Node for cloud functions, but it is something to think about.

Other FaaS providers don't have this requirement, and just take a Linux x86_64 binary.

View on HN

Indeed. If you want to compile the binary yourself/don't want to upload source code, we have a serverless containers product currently available as an early preview (sign up at g.co/serverlesscontainers). This would allow you to compile your binary locally, write a simple Dockerfile and then build/deploy the resulting container.

This is great. It sounds like what people were hoping would come from Zeit Now, though in their latest major release they moved away from serverless docker deployments. Their community was quite upset at that decision I think. I will have to check this out!

This is probably not a surprise for folks who have been using Python or Node for cloud functions, but it is something to think about.

I've been thinking of using this approach for a golang MMO game SaaS which is really an aggregation of a bunch of FaaS lambdas tied to an account. What upload allows you to do, is to do a syntactic scan of all of the code, particularly the package imports and function calls. All of the above can be whitelisted, which adds security. Also, you can munge the function calls against a generated munged version of the available libraries, which also adds security. (In addition to using containers under the covers.)

Like Screeps? Even more niche? :)

That looks cool.

What I'm describing is for implementing MMOs, though.

Curious, wouldn’t you want the source code to compile on the final platform? (If not using docker) I’m not familiar with binary or lower level workings but I figured there would be certain benefits of compiling the same code on Linux vs Mac or Windows for example.

The Go compiler produces the same binary regardless of which host OS it is ran on. i.e., the OS you compile a Go program on doesn't matter (assuming pure Go code, not including C etc).

I think he was talking more about the machine architecture than the OS.

As of Go 1.5 the compiler works for any supported target architecture.

Thx for the info, I don’t use Go typically.

I was wondering about this, given the examples. AWS's Golang support for Lambda involves including their library and passing your function to a Lambda initializer function. Which I'm sure is all that's happening with this, but it's weird to hand off the Go source to Google. I wonder if they are multiplexing multiple functions in single binaries or using special Go toolchains. Or just don't want to lock into specific backends yet.

No, there's no real magic. Each function runs in its own process (which is inside a gVisor sandbox).

The toolchain is completely unmodified.

Disclaimer: I work at GCP and worked on this product.

Unlikely Golang’s init block will get executed in all the multiplexed serverless functions that are bundled in the one binary. Which adds a side effect

> AWS's Golang support for Lambda involves including their library and passing your function to a Lambda initializer function

AWS announced 'native' Golang support last year - since then there hasn't been a need to use a Python or node 'shim' to bootstrap the golang executable like this. You just upload your compiled binary to S3 and provide the path to AWS lambda

That would be my guess. Since this is all event driven code, why wouldn’t they bundle them for improved system utilization?

Because Go's namespacing / packaging model doesn't have the kinds of isolation required to prevent you from accessing the code and variables of the other packages you've been bundled with, compared to NodeJS, which does (VM feature).

This is a much better solution, I think. Uploading your source code is often easier than uploading your binaries. If, for example, you have a new contributor you don't yet trust creating a cloud function in Go and you know enough Go to review the code, but it's not convenient for you to compile it yourself, you can just upload the code yourself. You can't easily review the binary.

The fact that it's easier to do things this way just shows how immature the toolchain surrounding serverless development is. It shouldn't be difficult to simulate the serverless environment on a workstation, and yet somehow that's a huge gap, even for Lambda.

Fair enough, but I don't think it's wrong to want a PaaS/FaaS that automagically compiles things. You'll be hard pressed to convince anyone who genuinely likes this approach to switch over to having their IDE, developer tools, or CI do it.

the Cloud Functions emulator works well, and helps https://cloud.google.com/functions/docs/emulator

Then again, what's the point of controlling your own repository of binaries if their only goal in life is to be deployed somewhere else anyway ? It's much easier to just cut the middleman.

Your code review and deployment pipeline should be all automated from day zero. Setting up a CI pays off in the first couple of days in terms of time investment and massive returns on longer term mostly for avoiding human error and confusions that ensues.

CI is a continuous integration tool. There are inherent latencies to committing, pushing, deploying to AWS, running tests. Mature development tooling, if it's not REPL-based, will at least allow a local debugger to step through the code compiled/run locally, without these remote latencies.

Both tools are necessary for fast engineering, but in different contexts.

I didn't say that you don't need CI. I use CI regularly, and I think it's best for CI to be for testing, and to minimize the build artifacts except those used for test results.

A lot of developers have this preference for client side code. Since the early days heroku has been building the client side bundles for rails.

The parent’s point was that contributors shouldn’t have permissions to upload binaries; they only commit source code and your CI infrastructure compiles, releases, and deploys it.

If i understand correctly the binary is created from the source code right. Am i missing anything?

On AWS Lambda, which has had Go support for almost a year now, you upload the binary, which you create from the source code on your own system, using the go compiler. On Google Cloud Functions, you upload the source code and Google's servers build it.

I don't really understand what you are saying here .. It should not be difficult to run `go build` locally and then tell a tool to upload `foobar` instead of `main.go`.

Uploading source code is a dealbreaker for me because I do not trust Google or myself. Google probably has language in their terms that they may do whatever they want with anything you upload to them.

I also do not trust myself to properly configure security parameters around this with the risk of leaking source code.

Binaries are a great strength of Go. Too bad this platform doesn't use them. I will take a peek at the new Container platform that they are building, but that sounds more heavyweight, which probably comes with a higher price. (I have not seen any actual pricing)

can't your security params be read out of the binary in any case? i'm not sure how binaries offer more security.

His post is super unclear but I think he is concerned about someone getting their hands on his source code. In general, a compiled Go binary is not trivial to reverse engineer, so there’s some security through obscurity there.

Not a whole lot, though. A determined reverse engineer will likely be able to grab something close to your source code without much difficulty, assuming you haven't applied any wacky obfuscation.

Not 'someone'. Google.

Your secrets are not safe by embedding them in a binary. Compilation is not (necessarily, and at least so in the Go compiler's case) security...

Reply on news.ycombinator.com