App Engine’s New Go 1.11 Runtime | Hacker News Reader

App Engine’s New Go 1.11 Runtime

blog.golang.org

168 pointsSamWhited96 comments

I'm the tech lead and primary software engineer on the Go runtime. If you have any questions, please don't hesitate to ask! I'm super thrilled to announce that Go 1.11 is now available on App Engine! We now support...

* vendoring

* regular best-practice package structures

* go modules

* the regular Google Cloud client libraries: https://github.com/GoogleCloudPlatform/google-cloud-go

This is a "second-generation" runtime (https://cloud.google.com/blog/products/gcp/introducing-app-e...), meaning that we're now running stock Go in the gVisor sandbox (https://github.com/google/gvisor). We've removed all of the restrictions present in the old runtime, like limited socket and file access. You can even import "unsafe"!

If you're a current Go-on-App Engine customer, you should check out our migration guide at https://cloud.google.com/appengine/docs/standard/go111/go-di... to learn how to migrate from the Go 1.9 runtime to the new Go 1.11 runtime. For the time being, you can still use the legacy App Engine APIs with the Go 1.11 runtime, but you should start migrating to the Google Cloud client libraries.

Yes! Thanks for releasing this and making my life easier :)

I have an old app deployed to App Engine. The app itself is rock solid and chugging along fine (I touch it once a year or so), but I dreaded having to deploy it due to the lack of vendoring support and other... peculiarities of the Go runtime. Glad to see this is no longer an issue!

Several of your links are broken, they have been elipsised.

The ellipsis in full is "go-differences" for the migration guide.

Sorry, I copy pasted without double checking. Now fixed!

I take it the datastore change is just an api change? (There isn't any data migration needed, is there?)

It looks like memcache is moving to a third-party service now instead of a built-in api. Is performance different? Does anyone have experience with this?

> I take it the datastore change is just an api change? (There isn't any data migration needed, is there?)

Yup!

> It looks like memcache is moving to a third-party service now instead of a built-in api. Is performance different? Does anyone have experience with this?

I'm not sure about the performance impact. But I can tell you that we're working with the Cloud Memorystore team to have a better memcache story for the App Engine second generation runtimes

Both google.golang.org/appengine/datastore and cloud.google.com/go/datastore connect to the same data? Is there a list of incompatibilities somewhere? I remember seing some serialization differences between these two datastore libraries.

Hi (App Engine PM here). Yes both libraries connect to the same database (Cloud Datastore). cloud.google.com/go/datastore is preferred as it uses the Cloud Datastore API instead of the App Engine specific API, so your code will be portable.

You are right that the devil is in the details, and there might be some slight serialization differences. I cannot find an exact list.

Feel free to post here if you need more help.

You remember right - I think you might be referring to this issue: https://groups.google.com/forum/#!topic/google-api-go-announ...

> the cloud datastore package will default to writing your nested structs as entity values, while the appengine datastore packages will only write your nested structs as flattened sets of attributes.

Other than that, there are no differences.

(GCP Gopher)

Does the move to gVisor pave the way to connecting CloudSQL instances? The removed limitations sound like they were road blocks.

you already could connect to cloudsql instances. however you needed a proxy, but not because of AppEngine, you need it because of security.

either you whitelisted your ips or you used the proxy (which go appengine could already use).

so with go you just needed to use https://github.com/GoogleCloudPlatform/cloudsql-proxy which means you just need to change the query string.

however a new feature is emerging where you can have a private ip that is connected to your internal services, however it is still a beta feature.

Source: myself, but not a googler, just a regular user

Notable, though, that this new runtime better supports connecting to Postgres instances through the unix socket (/cloudsql/).

(GCP Gopher)

For cloudsql in particular, you can connect via the `/cloudsql` Unix socket. But anything that needs raw socket access can now be used from App Engine.

See https://cloud.google.com/appengine/docs/standard/go111/using... and scroll way down to the sample code.

Awesome!

when will appengine support setting environment variables with gcloud command or inside the gcloud console instead of in the app.yaml, i.e. so that each environment can have their own variables without copying app.yaml's?!

(App Engine PM here) At this time, app.yaml is the only way to set env vars. If you want to separate environments, you could use different .yaml files, I personally use app.staging.yaml and app.prod.yaml

Not worried about secrets in repo?

Tech Lead of Google Registry ( https://registry.google ) here. We run on GCP.

You definitely don't want to check secrets into the repo for the same reason you don't set them as environment variables: It's not secure. The solution is to use Cloud Key Management Service. More info here: https://cloud.google.com/kms/ We use that to either store secrets directly, or to decrypt encrypted entities in a DB (e.g. Datastore).

You can see how we use it here (our project is open source): https://github.com/google/nomulus/tree/master/java/google/re...

Another great resource: https://sethvargo.com/secrets-in-serverless/

Shows how to encrypt env vars with Cloud KMS then decrypt them in memory.

This is rad. Thanks.

Neat, looks like they're using a solution similar to what we're doing, except instead of using Cloud KMS to encrypt information stored in Google Cloud Storage, we're putting our information in Datastore.

The reason being that these secrets are absolutely essential for many tasks that our registry needs to perform, and the past six years of experience have shown us that Datastore has better availability than GCS. We haven't seen Datastore ever go down unless all of Cloud is down too, whereas we have seen outages isolated to just GCS. Datastore also has lower latency (since it's a DB, not bulk file storage).

One caveat is that the maximum size of a single entity in Datastore is 1 MB -- if you're encrypting stuff larger than that then you'll need to shard (ugly) or just use GCS. Since none of our secrets are anything close to that large, it works just fine.

It’d be awesome if app engine had an easy way to add this like Firebase does.

P.s typo on your link

Also, the NYTimes just published their project which integrates with HashiCorp's Vault: https://open.nytimes.com/open-source-simplifying-serverless-...

> I'm the tech lead and primary software engineer on the Go runtime.

Hey thanks a lot for your efforts! Your direct communication about the Go runtime progress has been a breath of fresh air compared to the years preceding you. I wonder though, how many developers are there on the App Engine Standard Go team?

is gRPC supported now or is that more of an upstream HTTP 1.1 constraint?

Hi (App Engine PM here) You can use any go package in the Go 1.11 runtime, including the ones relying on GRPC. Did I answer your question?

You cant run a GRPC server though still right? Thats all I really want to be able to do especially now that the app engine endpoints libraries are no longer supported on python

Inbound GRPC will be limited to HTTP1.1; that's a limitation of the platform rather than the language runtime. We're working on HTTP2 support, but I can't promise any timelines.

By inbound GRPC you mean when running an app engine instance as a client right? So running a GRPC server is completely out of question still I would presume?

I mean running a GRPC server, yeah. It's a limitation of our networking stack which we're working on fixing.

It's always been my impression that Go on GAE is the most lightweight compared to Java/Python, as it not "memory hungry" like Java nor "slow" like Python. Am I correct?

It is quite lightweight, yes!

Though...I wouldn't necessarily call Python slow! Python keeps getting faster, and we now support Python 3.7 on our second gen runtimes: https://cloud.google.com/blog/products/gcp/introducing-app-e...

Does anyone have numbers to compare? Go is a compiled language. Should perform much better.

Python might be perfectly acceptable of course.

I would absolutely love those numbers too. Always wondered about the working set of a minimal program, and also one with database access.

Well, there's the Techempower benchmarks at: https://www.techempower.com/benchmarks/

You can go to the filter, turn off all languages but Go and Python, and get a general idea.

There's also the old Computer Benchmark Game, which should be taken with a whole barrel of salt: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

I actually looked at this before making my other comment. You’re absolutely right about the salt. One of the go benchmarks is basically pure cgo using vector intrinsics. It naturally blows idiomatic JS out of the water. So you might get those kind of results if that’s the kind of code you’re going to write, but it’s far from what I consider idiomatic. Another language’s proponent would be fair to point out that most languages have C FFIs.

> One of the go benchmarks is

Which program? URL?

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

It’s fifty percent faster than the competing java program.

That Go pi-digits program is 50% faster than a Java program that also uses the GMP library, just like the Go program!

That Go pi-digits program is only 4% faster than a PHP program that uses the GMP library.

It's not enough to "point out that most languages have C FFIs" when for this arbitrary precision arithmetic task GMP is allowed and others step-up and contribute those programs.

I see. I had only looked at the source code of the Nodejs competitor. To be fair the point still stands that you need to read the source to know what is really being measured.

Of course you need to read the source code to know what is really being measured!

That's true of every comparison -- basic due diligence.

And read how the measurements were made --

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

And read some background --

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

And, of course, look for a task that might be appropriate --

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

No, there's no reason to take the benchmarks game with even a pinch of salt!

The benchmarks game website shows what it says it shows --

"Which programs are faster? Will your toy benchmark program be faster if you write it in a different programming language? It depends how you write it!"

(No, there's no reason to add even a pinch of salt to the benchmarks game website).

Nodejs might also be a good option. It’s comparable to go on a core for core basis, and IIRC GOMAXPROCS is 1 on GAE.

(GAE Gopher)

GOMAXPROCS is not 1 for this new runtime. As far as I know, it's an entirely vanilla Go runtime.

Woah that's a significant update then.

Any plans to support the cloud endpoints with the standard env? Wondering if the new runtime would be able to be embed that. That would be only reason for me not to switch from flex env.

(GAE PM)

Cloud Endpoints currently runs on GAE Standard Gen 1 and GAE Flex. We're investigating a solution for GAE Standard Gen 2 (which go 1.11 is, along with python 3.7, node 8, etc.) and GCF.

Thanks for the clarification. Wasn’t aware that it works with Go on Standard Gen 1.

Any ETA for Gen 2 or alpha signup?

Clarification: GAE Standard uses the Endpoints framework, which is Python and Java only (https://cloud.google.com/endpoints/docs/frameworks/about-clo...). Sorry for the confusion.

Flex uses ESP (https://github.com/cloudendpoints/esp), which is deployed as a sidecar, and works with any runtime.

As for EAP: unfortunately not. Personally, I'd like to see a managed version as opposed to the current framework or sidecar approach, as it'll be easier to implement across products, but that means it'll likely take a little longer.

How long will the legacy App Engine APIs continue to work? Will they be removed before the runtime is GA?

These new runtimes seem amazing (and fix a lot of problems), but we also lose a ton of functionality that made app engine so desirable (easy users auth, images api, built in email sending, search, cron, and others).

It kind of feels like we've thrown the baby out with the bathwater.

(App Engine PM here) The App Engine APIs will be present in the Go 1.11 runtime when it goes GA and until it is turned down. We do not know exactly which future version of the runtime will stop supporting them.

We are working on Google Cloud standalone products to replace most missing features (e.g. Cloud Scheduler for cron jobs). Not all of them are ready yet.

App Engine APIs were great when there was no alternative (when GCP did not exist, or when App Engine could not use arbitrary packages), but at the same time, contributed to the "lock-in" of App Engine, which was one of the main criticism. We believe it is in the long term benefit of our users to use standalone Google Cloud or third party services, instead of replying on APIs only accessible in App Engine.

Thanks for the explanation. It makes sense and hopefully replacements are generally available before we lose the old apis.

Hey buss, thanks for stopping by.

How do you feel about the distinction between Google and Go? It sometimes feels like they're joined at the hip, something which historically had made people queasy (Java and Oracle, .NET and Microsoft, etc). Though Go is a project which came from Google, I personally feel that it better serves the interests of the Go community to treat them separately. Google receives special treatment from Go - the announcement that AWS Lambda would support Go did not receive similar fanfare on golang.org.

Side note: there is some activity on the Go blog which is more mutual cooperation and less Google hivemind:

https://blog.golang.org/go-cloud

But I still wonder why this stuff belongs here.

Agreed. To even submit content to the Go Blog requires a Google account. Ugh.

Three smart guys made go. Google only has one or two left.

I just tried to use go111, but I get the following message:

>[7] Access Not Configured. Cloud Build has not been used in project <project> before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudbuild.go... then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

So go111 cannot be used without Cloud Build, for which I have to activate billing?

Ah, yeah, looks like you have to activate billing. You'll be solidly within the free tier, though, so you won't be charged.

You can set a $0 limit on your billing account, too.

I thought Go was statically compiled....i.e. it doesn't have a separate runtime from the program.

Does anyone have any elucidation?

Correct, Go's runtime is compiled into the program's binary.

However, there's still an environment needed to run that binary. For Go on App Engine, this includes the sandbox (backed by gVisor) and the operating system (Ubuntu).

We also tend to call everything in the toolchain the "runtime" - this includes the CLI for uploading and staging your app, and the builder used to compile your program. Not the strictest definition :)

(GCP Gopher)

So App Engine is a build system as well?

Yes, with App Engine, you upload source code, and it's compiled for you remotely. For this newer runtime, it happens inside a Cloud Build invocation.

For App Engine flexible, you can skip that build step and provide your own container, but that isn't possible for App Engine standard. (If you're interested in something like that, see https://g.co/serverlesscontainers)

Vendoring! Modules! Finally. Do you have any examples with proposed solutions for local module setups, without having to rely on GOPATH changes?

This is a go question, not an app engine question. See this page on the go wiki for an example of how to use go modules: https://github.com/kubernetes/kubernetes/pull/58098

Note: you need to have go1.11, and by default you need to be in a directory NOT in your $GOPATH for modules to work in go1.11

Edit: https://github.com/golang/go/wiki/Modules#how-to-use-modules

Copy paste error I guess.

Does context.background() works ?

(GCP Gopher)

Yes, you don't need to use the appengine context for anything (except to access the services at google.golang.org/appengine/...)

You can use the standard net/http package to make HTTP requests, for example.

This is a big improvement, but the restriction that prevents streaming responses unfortunately still apply: https://cloud.google.com/appengine/docs/standard/go/how-requ...

Would be great if that was solved.

I'm not trying to stir shit up...buuut it seems odd that an announcement for a Google Cloud product is being published on the Go project's blog?

I say this as a huge fan of Google Cloud and the experience of using Go on App Engine.

Likely because it's clearly Go related?

"Announcing [some cloud]’s New Go 1.11 Runtime"

That being said, I have no idea how similar submissions of Go content from competing vendors would er... go. ;)

Does this new classic+ environment now supersede the Flex environment?

I’m curious because now it’s seems like there are 3 solutions. Although I really like the simplicity of this solution and the modernizing from the constraints of true classic mode.

Eh, not exactly supersede, but the runtimes are definitely slowly converging. It's a goal of mine to make them easy to switch between -- sometimes you'll want flex because you need a VM, most times you wont. But your code should be easily portable between them. With this release, they are nearly identical already.

So two of the solutions are flex and go1.11 on standard. Is the third solution the older runtime versions (eg 1.9 and earlier)? If so, then there's really just two solutions, since 1.11 replaces 1.9 on standard.

(GCP Gopher here)

If you can run within the standard environment, I'd suggest you do (but you should check pricing, etc to see if it suits). Especially if you want 0-1 scaling.

I migrated golang.org from standard to flexible a couple weeks ago, primarily because of increased control over the runtime (custom Docker image) and increased instance sizes (configurable memory, CPU past the App Engine limits, which caps out at 2 GB). See https://golang.org/issues/27205 if you want to read more.

Do you have blog post with data, number of instances etc previously and now ?

I don't, sorry. While the app was unhealthy, it was spinning up dozens of instances. While it was healthy (on standard), I recall it ran on less than a handful of instances.

It currently runs on two fat flexible instances (GCE VM), but could easily run on one. We only run on two for some redundancy.

I've measured throughput of ~thousands of requests per second on a F4_1G on apps I've worked on previously. They were CPU-constrained, but not doing very much (JSON serialization).

The new runtime might be even better, since GOMAXPROCS is no longer 1.

correction to my post above: 1 GB is the max memory for standard: https://cloud.google.com/appengine/docs/standard/#instance_c...

I'm loving the new vendoring via go modules. Can't wait for it to be the default in Go 1.12

Before anyone asks, this is still a beta release that's not covered by App Engine SLA.

Wonder if it's still possible to get answers here :) giving it a shot although it's 13h old by now.

We use Go for a production server for multiple of our games, it's used to sync gamestates between devices and allows us to remotely inspect and change player's gamestate. Super handy and it's worked flawlessly! We used to use python, but Go is much better suited as it's less error prone.

One question though, we've been having connectivity issues in China, the GAE is deployed in the US, and as it seems there's no multiregional support. What can we do? Ideally keeping as much of the convenience of the GAE as possible.

This actually seemed to be live a week or two ago, I managed to run Miniflux [1] on App Engine with no changes though to get it working properly it would need it's scheduling changing.

[1] - https://github.com/miniflux/miniflux

(App Engine PM here) Nice to hear! Yes indeed, we announced Go 1.11 on App Engine last week at Cloud Next London.

Ah I hadn't seen that, I was quite surprised when I used goapp111 and it deployed successfully. It's great to be able to take a normal Go application and run it on Google Cloud. Thanks!

Is this a GA product or still "beta"? Google's consistent announcement of products as available while in beta is getting annoying, because their SLAs for "beta" products are still 0 and thus impossible to be able to use in actual business applications.

(GCP Gopher)

Yes, it's beta.

You're right - if you require an SLA, don't use beta products.

As per https://cloud.google.com/terms/launch-stages:

> Beta is the point at which we are ready to open a release for any customer to use. There are no SLA or technical support obligations in a Beta release, and charges may be waived in some cases. Products will be complete from a feature perspective, but may have some open outstanding issues. Beta releases are suitable for limited production use cases.

Hmm, I was just running into some context annoyances but this seems to remedy some nuances regarding that. Cool! Good timing!

I’m assuming urlfetcher is deprecated now too? I’ll go read the migration guide now ...

Awesome work! Love Go and GAE! Very happy so far.

You can use net/http to fetch URLs now :)

Glad you're enjoying both Go and GAE!

What was the reason urlfetcher had to be engineered in the past? Curious.

I don't think I can say much more than: technical choices that required a different path to fetch resources from the broader internet.

Am I understanding this correctly, App Engine has made it so "go run goapp.go" on my local environment is basically accomplished with "runtime: goapp" in the app.yaml? No other changes necessary in the code?

runtime: go111, but otherwise -- yes!

I haven't used GAE since early days, but I heard it supports Docker now. Why wouldn't I just use Docker with whatever runtime I want?

(GCP Gopher)

We support custom images for the flexible runtime, but not for the standard runtime.

If you're as keen for that as I am, sign up here: http://g.co/serverlesscontainers

I have some C++ libraries called from Go - can I use them now?

cgo is supported.

Reply on news.ycombinator.com