Hacker News Reader

Comment by flakiness | Hacker News Reader

The post should be marked as [2006]. A lot has changed since then. For example it doesn't talk about NUMA but it should, if it were written today.

I would love to hear some opinions from the authors of more contemporary web servers like one of H2O or Envoy. (The latter may have slightly different use cases though.)

View on HN

Yes, 2006 would be a more proper tag.

NUMA is interesting, I've had a lot of fun with it on a system to control the adaptive mirrors in ESO's ELT telescope, and it is /amazing/ what you can do with a modern server, even on a vanilla kernel.

However, it is a lot harder to exploit NUMA in an application like Varnish, because the requests arrive as a randomized stream, and the effort to figure out which NUMA pool is best situated to handle the request, is not significantly different from just handling the request to begin with.

We used to serve a cache hit in seven system calls, an accept(2), a read(2), a write(2) and four timestamps, and as a result, Varnish servers ran with CPU's >70% idle.

If we had added NUMA-awareness on top of that, we would on average Nnuma-1/Nnuma of the time add at two system-calls to migrate the request to another NUMA domain, and that made little sense.

Since then NICs have become NUMA aware and since we use scatter/gather I/O, that matters a lot more than which CPU core did the overhead processing.

Where NUMA may become mandatory is 100G and 400G networking, but very few people seem interested in running that much traffic through a single server, for reasons of reliability, but I'm keeping an eye on it.

Checkout TCP_REUSPORT_LB_NUMA on FreeBSD.

It will filter incoming TCP connections to listen sockets owned by threads bound to the same NUMA domain as the NIC that received the packets. This is part of the Netflix FreeBSD NUMA work described here https://papers.freebsd.org/2019/eurobsdcon/gallatin-numa_opt...

The idea is that you can keep connections local to a NUMA domain. We (Netflix) have a local patch to nginx to support this (basically just setting the socket ioctl on the listen socket after the nginx worker is bound)

Yeah, so there are a lot of things we could optimize for on different kernels, but we prefer to not do so, until we have a really good reason.

As I said above, it is not obvious to me that we would gain much over what we already do, but I am keeping an eye on it.

Why did the timestamps require syscalls / do they still do now? (I know CPU counters were a mess back then, but was there any other factor about that?)

Back then they did.

But it is one of those things were I just chose to trust the OS to DTRT, and automatically reaped the benefits of advances "under the hood".

Well if by contemporary you mean written by Google builds with Bazel and runs in Docker, stuff like Envoy is pretty good and it uses Joyent's HTTP parser which is more readable than the pigeon C parsing Varnish likes to do. But Varnish understands UNIX much better than Envoy does, as evidenced by its portability and the things you've read in the blog post. Compared to Squid then hands down Varnish is the best. I'm not sure if it's better than open source GFE but it's pretty good.

None of them go as fast as redbean of course. The blog post talks about how they only need 18 system calls to serve a local asset. redbean only needs one system call: writev(). That's because redbean cheats by having a novel design. The zip central directory means we don't need file system system calls. The invariant RDTSC instruction means we can also have timestamps without context switching.

Even for 2006, what would make this argument for me is actual numbers. Is it plausible that some programs worked well like that then? Sure. But I can promise not everything did. The kernel's approach to caching is optimized for the general case, but each program is specific. It's plausible to me that a caching service like Varnish is a very close match. But that's an empirical question, not a theoretical one.

Forgive my ignorance, but doesn't http://varnish-cache.org/docs/trunk/phk/notes.html#more-cach... talk about that, with an example?

It's about cache vs memory. NUMA is about local memory vs remote memory. So similar but different IMO.

What's different about NUMA today vs 2006?

I would say nothing, except maybe for how many mentions it gets in HW-review articles.

Unless you are in numerical HPC or kernel programming, you do not need to think about NUMA, but it is not black/white, it is a slope.

Varnish tries to do certain things "NUMA-friendly", without actually being "NUMA-aware" as such, and the kernels seem to do a really good job with that.

NUMA was not widely available / understood in 2006. I don’t recall seeing my first NUMA servers until probably 2008ish.

That makes sense to me for the entire computing landscape, like if we're including windows, various 32-bit x86 unix like OS's, etc. But varnish was used on Solaris, IRIX, and HP/UX. All of which had cc-NUMA support fairly well established by 2006. I suppose, though, it was still new-ish.

First Opterons were released in 2003

Envoy is not a web server.

Why not? And what’s the definition of a „web server“ anyway?

In my opinion it’s about whether a software can process http requests on a configurable port - and envoy can for sure do that.

A webserver, as bare minimum, should be able to serve static files from the disk - but you are correct, there is no such definition, so this is an opinion.

Regardless, calling envoy a webserver is misleading: it's both more and less, than a "traditional" webserver. It's a proxy; call it a proxy, please.

> A webserver, as bare minimum, should be able to serve static files from the disk.

You can configure routes with a `direct_response` handler from envoy.yaml that is read from disk ;)

It's a function from request to response.

Reply on news.ycombinator.com