Hacker News Reader

Comment by gtrubetskoy | Hacker News Reader

There's still one thing missing - graceful restart:

https://grisha.org/blog/2014/06/03/graceful-restart-in-golan...

BTW - to people new to Go this article may make it look like serving HTTP is complicated, but it's actually remarkably easy. And if you consider that it is actually _possible_ to have a complete server with just the standard lib (and with TLS and HTTP/2 to boot) running as a single process - compared to Python or Ruby, where because of the GIL you _must_ place an apache/nginx/haproxy in front of it and also run a bunch of unicorns or something similar on different ports (at which point you need something like Chef/Puppet to manage the config because it gets very complicated very fast) - this is actually pretty amazing.

View on HN

You mean missing from the article or from Go stdlib? This was added in Go 1.8 (in beta now), so that link is a bit out of date I think.

https://github.com/golang/go/issues/4674

Agree that serving http/https with Go is really pretty straightforward - you can do it without all the tweaks in this article, and most people have been. It works remarkably well by default and scales well too without much effort.

I meant missing from both, but I didn't realize that it's making its way into the standard lib - this is very cool, thanks for the link!

> compared to Python or Ruby, where because of the GIL you _must_ place an apache/nginx/haproxy in front of it and also run a bunch of unicorns or something similar on different ports

or you know, just call fork() a few times, like Apache has done for more than a decade

This is precisely what unicorn is meant to do though, it's just a trope of the architecture to use hproxy on top.

Calling fork() internally is ultimately akin to what everyone is going to do. Why not use a library to do this and let that code gradually get tempered to a reliable status?

Isnt that a lot more overhead than green threads in a single proc?

It doesn't matter if you have work outside of I/O to do for request responses.

And hope you don't have any state that can collide.

fork() starts new processes, so that would be a weird state to collide.

I think you mean that if you are calling fork you better not have any state. Otherwise you will soon have multiple independent/mutated copies of your state which is normally not a good thing. But you are correct that they would not collide.

We use SO_REUSEPORT [1] to implement graceful restarts. There's a library for go that allows you to do that [2].

1. https://lwn.net/Articles/542629/

2. https://github.com/kavu/go_reuseport

Example of code using SO_REUSEPORT. Ghostunnel does it to reload short lived certificates: https://github.com/square/ghostunnel/blob/master/main.go#L30...

How does it do that without dropping open connections (does it drain)?

Here's how it works in our case:

1. Start a new process, start accepting new connections

2. In your old process, stop accepting new connections — old ones are still active.

3. In the old process, close old connections when they become idle.

4. In the old process, wait until all of your connections are closed and then quit or automatically quit after a timeout.

You can close the socket without dropping existing connections.

As has been pointed in some of the other subthreads, another reason to run nginx in front of Go/Python/Ruby is that running on 80 or 443 needs root access. From a deep security perspective, it's better to run your app as a dedicated user with only the necessary privileges.

Also, assuming that any static assets are served, probably a good idea to leverage sendfile.

>running on 80 or 443 needs root access

Not necessarily:

    setcap 'cap_net_bind_service=+ep' your_go_binary
    ./your_go_binary

This interesting, I have never seen setcap. It seems it doesn't work with scripts (ruby, python) and if you are using JVM/mono/beam you will need to setcap the whole VM, but a very cool solution for a language like Go with binaries!

My two cents, you probably need to apply setcap in Python interpreter itself instead of the script. It shouldn't be a problem though, since you probably will use a virtualenv anyway.

Another option would be to drop privileges at runtime.

>It shouldn't be a problem though, since you probably will use a virtualenv anyway.

virtualenvs don't create a new interpreter, they just fudge the python path?

Definitely not recommended on interpreted languages (although we use it all the time on our go apps).

They create a copy of the binary of the interpreter, you can even call it directly instead of activating the virtualenv first.

If you have a service run with systemd you can use this in the unit file:

AmbientCapabilities=CAP_NET_BIND_SERVICE

otherwise you'll have to run setcap any time the binary changes.

> Also, assuming that any static assets are served, probably a good idea to leverage sendfile.

Go already leverages sendfile:

https://golang.org/src/net/tcpsock_posix.go#L44

Go HTTP standard library does use sendfile whenever possible (you need some kinds of file descriptors, etc...)

And why is nginx safer?

Because, nginx is a hardened program used by thousands of companies with a strong interest in making sure there aren't vulnerabilities. Also, it is a program with a very narrow set of functions. Nginx vulnerabilities are comparatively rare: two security advisories in 2016, none in 2015. [0]

By comparison, app code is often developed rapidly and often only reviewed by at most a few people. Even companies staffed by brilliant minds like Google regularly have vulnerabilities in their application code.

0. http://nginx.org/en/security_advisories.html

It also takes somethings that would suck to learn the ruby version of, the php version of, the node version of, the python version of, etc., and does it faster, and allows you to isolate it from your application's complexity and business logic cleanly. Those things are serving static assests, SSL, and backend spliting. I cringe everytime I see a tutorial loading up the framework in a slow language like node or php to pass a static asset without the use of something like Apache or nginx.

Multi-threading http servers in Python like CherryPy has a decent performance regardless of GIL.

People need to stop hype the buzzword GIL. NodeJS/GoLang doesnt even support native threading.

Go does not provide direct access to threads, but multiple goroutines can run on multiple threads simultaneously. Go is not single-threaded like Node.js nor "only one thread can run at any one point in time" like CPython. Go's concurrency provides real parallelism.

Go provides real parallelism if run on multiple CPUs, otherwise things are running concurrently (a la Python).

Of course. It's impossible to do parallel computation on a single CPU.

> but multiple goroutines can run on multiple threads simultaneously

Is goroutines very effective with CPU-bound computation tasks?

I don't know if it's "very effective" (compared to what?), but it's fine to do CPU-bound computation in goroutines. The Go runtime will try to schedule goroutines to run in parallel on different threads.

What about goroutines?

I never even considered the possibility. We take ours down so rarely (months in between) that a couple seconds of failed requests are fine. I understand however there are circumstances where that may not be the case.

Thanks for this article. Do you find that incrementing the waitgroup on every accept tends to become a bottleneck? Or are you siloing your server on a single core or something like that?

It should be negligible. It's an atomic add, and the OS accept function call should also be much more heavy weight. Add to that lots of synchronization that will happen on all further socket HTTP/socket reads/writes, especially if it's HTTP/2.

> Do you find that incrementing the waitgroup on every accept tends to become a bottleneck?

The specific thing I was dealing with when I wrote the blog post wasn't high-volume enough for it to make any difference, so I don't really know....

But is there a better option than a WaitGroup here?

Check out the Context API, it's a standardization of several cancellation approaches.

Those two go hand in hand. Even if you you context (or a bare channel) for signaling the cancellation you often want to wait until the cancellation really happened and the child task terminated. For this task WaitGroup is the easiest thing to use.

And x/sync/errgroup if you want the lovechild of context and WaitGroup

Python can self host. I've heard great things about CherryPy which is actually used to deploy things like Flask, even though CherryPy can work by itself.

You argue that people want to put Go in front whereas you would not with Python/Ruby because of the GIL. This is a misunderstanding because people use nginx in front for failover and load balancing, and not just because Python/Ruby are comparatively slow. So your argument is fundamentally flawed.

Python/Ruby is single process, and to exploit all the resources on a single machine you need multiple processes of your app (usually #cores or #cores-1).

Now that you have multiple processes, you need something to dispatch between them - intro Nginx/Gunicorn etc.

A single Go process can exploit all the system resources available without the multi-process orchestration required by Python/Ruby and thus does not require the extra layer above.

> Python/Ruby is single process

That's an implementation detail of your Python/Ruby program. I've written plenty of Ruby apps that have been multi-process and multi-threaded, and both at the same time.

> Now that you have multiple processes, you need something to dispatch between them

Yes, that's the case irrespective of language.

> A single Go process can exploit all the system resources available without the multi-process orchestration required by Python/Ruby and thus does not require the extra layer above.

This is no different than for Ruby at least.

Your criticisms are mostly outdated as of Ruby 1.9.x, though you can get bitten by the GIL. (EDIT: to clarify, avoiding this depends on either multi-process or taking advantage of the fact that while Ruby Thread's can only be scheduled one at a time, C-extensions etc. that releases the GIL can run in parallel with other Ruby Thread's; of course all of this is MRI specific in any case)

In practice, if you use a reasonably modern web server like Puma, you get good concurrency without having to think much about it.

>> Yes, that's the case irrespective of language.

I don't need gunicorn or nginx in front of my Java/Go/C# process to achieve concurrency and take advantage of all CPU cores.

>> Your criticisms are mostly outdated as of Ruby 1.9.x

Python 3.x & Ruby 1.9.x have the same limitations - any native code will lock the GIL, so unless all your code is in extensions you're going to run into this problem. Ruby 1.9 introducing OS threads did not change this.

Sure it's great that I can write an extension in C and take advantage of multiple OS threads in my Ruby/Python runtime, but the moment those threads have to deal with ANY native objects you're back to square one.

Your single Go Process is typically #cores Processes. All the orchestration is still handled by you, just there falacities for it in the language for it.

Just because you are using stdlib not an external application does not change the functionality that is happening.

I think this is a bit disingenuous.

Do you also think that Java is multi-process? I can see the argument that an OS thread is a process but it is basically irrelevant in every way that matters in development and operations.

An example - how do you access the same shared-memory (say, a map/dictionary) from multiple cores in any particular runtime?

- Go: you make a map, you protect the map with a mutex, and you reference it directly.

- Java: same, or ConcurrentHashMap or whatever is available.

- 'Worker'/'forking' runtimes: you can't, right? You move the state to another component (such as Redis), or you use an OS-provided shared memory facility, or you need inter-process RPC etc ...

Of course, sometimes workers being separate processes is a virtue rather then a burden ... but I'm not sure this is true in the case of Ruby or Python when it's more purely a limitation of the runtime.

It orchestrates using threads iirc, and the primitives for handling the communication in a more safe fashion are in the box, as you mention... and depending on the OS starting threads may be nearly as expensive as processes. That said, it's definitely less overhead than orchestration across forked threads.

There's multiple layers of "in front". Python and ruby shops often have dedicated L7 load balancers, and then run nginx on individual hosts to mux to multiple dynamic language processes/threads.

Reply on news.ycombinator.com