Hacker News Reader

Comment by ctxcode | Hacker News Reader

These kind of tactics work for simple examples. In real world http servers you'll retain memory across requests (caches) and you'll need a way to handle blocking io. That's why most commonly we use GC'd/ownership languages for this + things like goroutines/tokio/etc.. web devs dont want to deal with memory themselfs.

View on HN

It scales to complex examples as well. Retained memory would be handled with its own allocator: for a large data structure like an LRU cache, one would initialize it with a pointer to the allocator, and use that internally to manage the memory.

Blocking (or rather, non-blocking, which is clearly what you meant) IO is a different story. Zig had an async system, but it had problems and got removed a couple point releases ago. There's libxev[0] for evented programs, from Mitchell Hashimoto. It's not mature yet but it offers a good solution to single-threaded concurrency and non-blocking IO.

I don't think Zig is the best choice for multithreaded programs, however, unless they're carefully engineered to share little to no memory (using message passing, for instance). You'd have to take care of locking and atomic ops manually, and unlike memory bugs, Zig doesn't have a lot of built-in support for catching problems with that.

A language with manual memory allocation isn't going to be the language of choice for writing web servers, for pretty obvious reasons. But for an application like squeezing the best performance out of a resource-constrained environment, the tradeoffs start to make sense.

[0]: https://github.com/mitchellh/libxev

Off the top of my head, I was wondering... for software like web services, isn't it easier and faster to use a bump allocator per request, and release the whole block at the end of it? Assuming the number of concurrent requests/memory usage is known and you don't expect any massive spike.

I am working on an actor language kernel, and was thinking of adopting the same strategy, i.e. using a very naive bump allocator per actor, with the idea that many actors die pretty quickly so you don't have to pay for the cost of GC most of the time. You can run the GC after a certain threshold of memory usage.

Have you looked at how Erlang does memory management within its processes? You definitely can "get away" with a lot of things when you have actors you can reasonably expect will be small scale, if you are absolutely sure their data dies with them.

The trick to Erlang's memory management is that data is immutable and never shared, so all the complication and contention around GC and atomic locks just disappear.

The key thing (as I understand it) is that each process naturally has a relatively small private set of data, so Erlang can use a stop-the-process semispace copying collection strategy and it's fast enough to work out fine.

Since nothing can be writing to it during that anyway, I'm not sure the language level immutability makes a lot of difference to GC itself.

If you never cache any data. Sure, u can use a bump allocator. Otherwise it gets tricky. I havent worked with actors really, but from the looks of it, it seems like they would create alot of bottlenecks compared to coroutines. And it would probably throw all your bump allocator performance benefits out the window. As for the GC thing. You cant 'just' call a GC.. Either you use a bump allocator or you use a GC. Your GC cant steal objects from your bump allocator. It can copy it... but then the reference changes and that's a big problem.

I think this comment assumes that you're using one allocator, but it's probably normal in Zig to use one allocator for your caches, and another allocator for your per-request state, with one instance of the latter sort of allocator for each execution context that handles requests (probably coroutines). So you can just have both, and the stuff that can go in the bump allocator does, and concurrent requests don't step on each others toes.

The problem _somebody_ between the hardware and your webapp has to deal with is fragmentation, and it's especially annoying with requests which don't evenly consume RAM. Your OS can map pages around that problem, but it's cheaper to have a contiguous right-sized allocation which you never re-initialize.

Assuming the number of concurrent requests is known and they have bounded memory usage (the latter is application-dependant, the former can be emulated by 503-erroring excess requests, or something trickier if clients handle that poorly), yeah, just splay a bunch of bump allocators evenly throughout RAM, and don't worry about the details. It's not much faster though. The steady state for reset-arenas is that they're all right-sized contiguous bump allocators. Using that strategy, arenas are a negligible contribution to the costs of a 200k QPS/core service.

This example came from a real world http server. Admittedly, Zig's "web dev" community is small, but we're trying :) I'm sure a lot could be improved in httpz, but it's filling a gap.

You can use these patterns for per-request resources that persist across some I/O calls using async if you are on an old version of Zig or using zigcoro while you wait for the feature to return to the language. zigcoro's API is designed to make the eventual transition back to language-level async easy.

Yes, we were creating Apache and IIS plugins 25 years ago for PHP, Perl, Tcl, Coldfusion, ASP (scripting COM in VB), or C++ frameworks like ATLServer.

Very few were dealing with raw memory management in C, without anything else.

And all of this evolved into the Web application servers, and distributed computing landscape of modern times.

Reply on news.ycombinator.com