Leveraging Zig's Allocators
openmymind.net
openmymind.net
BTW, I used zig a lot recently and the opaque allocator system is great. You can create weird wrappers and stuff.
For example, the standard library json parser will parse json, deserialize a type that you requested (say, a struct). But it needs to allocate stuff. So it creates an arena for that specific operation and returns a wrapper that has a `deinit` method. Calling it deinits the arena so you essentially free everything in your graph of structs, arrays etc. And since it receives an upstream allocator for the arena, you could pass in any allocator. A fixed stack allocator if you wish to use stack space, another arena, maybe jemalloc wrapper. A test allocator that checks for memory leaks.. Whatever.
I assume the receiver then has to know it has to clone all of those values, yes?
That seems a little tricky for general code and moreso for unit tests.
> I assume the receiver then has to know it has to clone all of those values, yes?
The receiver needs to understand the lifetime any which way. If you parse a large JSON blob and wish to retain arbitrary key/values you have to understand how long they're valid for.
If you're using a garbage collection language you can not worry about it (you just have to worry about other things!). You can think about it less if the key/values are ref-counted. But for most C-like language implementations you probably have to retain either the entire parsed structure or clone the key/values you care about.
I have stopped considering this sort of thing as a potential addition to the language because the BDFL doesn't like it. So realistically we must remember to write reset, or defer deinit, etc. This sort of case hurts a little, but people who are used to RAII will experience more pain in cases where they want to return the value or store it somewhere and some other code gains responsibility for deinitializing it eventually.
What I would like to see would be some way of forcing users to manually call the dtor/drop. So when I use a type like that I have to manually decide when it gets destroyed, and have actual compile checks that I do destroy the object in every code path.
Not so in Zig: whenever you deal with collections where either the keys or values manages a resource (i.e. does not have value semantics), you have to be incredibly careful, because you have to consider lifetimes of the HashMap and the keys/values separately. A function like HashMap.put is sort of terrifying for this reason, very easy to create a memory leak.
I get why Zig does it though, and I don't think adding C++/Rust style value semantics into the language is a good idea. But it certainly sometimes makes it more challenging to work in.
I think the latter problem is impossible, you end up needing RAII or a type-level lifetime system that can't express a lot of correct programs. I would like something that prevents "accidentally didn't deinit" in simple cases, but it probably wouldn't prevent "accidentally didn't call reset on your FixedBufferAllocator" because the tooling doesn't know about your request lifecycle.
Even if you don't have process isolation between workers (which is generally what you want) then you can still put memory arenas far apart in virtual memory, make use of inaccessible guard pages, and take other precautions to prevent catastrophic memory corruption.
And that handler signature would still be the same. Which is the he whole point of this article so, yay.
Finally figured it out: I flipped a sign in the reverse iterator, so it was allocating a bunch of memory and immediately hitting the margin of the Vector, and returning it with most of the bytes undefined. Why didn't I catch it sooner? Well, I kept running the benchmark, which allocated a reverse buffer for the String version, which GC released, then I ran the buggy code... and the GC picked up the recently freed correct data and handed it back to me! Oops.
Of course, if you want to avoid that risk in Zig, you just write a ZeroOnFreeAllocator, which zeros out your memory when you free it. It's a drop in replacement for anything which needs an allocator, job done.
The current alloc implementation memsets under the hood. There are ongoing discussions about the right way to remove that performance overhead, but safety comes first.
Any sane implementation has an arena per request and per connection anyway, not shared between processes. You don't have bonkers aliasing bugs because the OS would have panicked before handing out that memory.
Zig has a lot of small features designed to make memory corruption an unhappy code path. I've had one corruption bug out of a lot of Zig code the last few years. It was from a misunderstanding of async (a classic stack pointer leak disguised by a confusion over async syntax). It's not an issue since async is gone from the language, and that sort of thing is normally turned into a compiler error anyway as soon as somebody reports it.
If you need data to be isolated when memory is corrupt, you need it to be isolated always.
If we use a modern Xeon chip as an example, an AVX2 store has a throughput of 2 instructions / cycle. Doing that 256 times for 8 KB totals 128 cycles, plus a few extra cycles to account for the latency of issuing the first instruction and the last store to the L1 cache. With a 2 GHz clock frequency, it still takes less than 70 nanoseconds. For comparison, an integer divide has a worst-case latency of 90ish cycles, or 45ish nanoseconds.
Zeroing memory on deallocation can be important for sensitive data. Otherwise, it makes more sense to zero on allocation if you know that it's needed because the allocated structure will be used without initilazation and the memory isn't zero by guarantee (most OSes guarantee newly allocated memory will be zero, and have a process to zero pages in the background when possible)
If you're exposing an HTTP API to clients, it's likely that any response's contents will contain sensitive client-specific data. If memory corruption bugs are more likely than bottlenecking on zeroing out your request/response buffer, then zeroing the request/response buffer is a good idea, until proven otherwise by benchmarks or profiling.
If it was freed after already being removed from the L1 cache, then you also need to evict other L1 cache contents and wait for it to be read into L1 so you can write to it.
128 cycles is a generous estimate, and ignores the costs to the rest of the program.
But yes, it can be an over-optimization.
Memcpy, bzero and friends are insanely fast. Practically free when those bytes are in the cpu’s cache already.
(But it also has a 16KB page size.)
(By the way, x86 does have a dedicated instruction--rep stosb--but compilers differ as to how often they use it, for the reason cited above.)
Arm64 provides `dc zva` for this.
There are various _s variants of memset, etc that require the compiler to perform the operations even if it “proves” the data cannot be read.
And finally modern hardware has mechanisms to say “this is now zero” and not actually zero the memory and instead just tell the MMU that the region is now zero (which removes the cpu time and cache impact of accessing the memory directly).
On macOS and iOS I believe all memory is now zero’d on free and I think malloc ostensibly therefore guarantees zero’d memory (the problem I think is whether calloc tries to rely on that behavior, because then calloc can produce non-zero memory courtesy of a buffer overrun/UaF after free has ostensibly zero’d memory)
In Linux you mean.
Generally, no paper or SO answer will tell you where your program spends its time. Learn to use profiling tools, and experiment with stuff like this. Try out arenas. Benchmark before and after and see what kind of real performance difference it makes in your own program.
You might have 1 fixed buffer, for N (500+) ArenaAllocators (but only being used per one at a time). This allows you to allocate a relatively large fixed buffer since you have relatively few threads.
If you just used retain_with_limit, then you'd either have to have a much smaller retained size, or you'd need a lot more memory.
https://github.com/karlseguin/http.zig/blob/c8b04e3fef5abf32...
Matklad also had an interesting take in one of their blogs: if you have to write unsafe code, then Zig is actually both a more ergonomic language than Rust, and easier to write "safe" unsafe code in.
It is likely that zig will solve this. Rust says you must solve this with the type system. I'm not convinced that is the case.
And I honestly think I'd have completely stopped writing code if I hadn't discovered Zig, after Rust made me hate what I used to love.
Our profession's tendency to take the easy way out on tech choices is eventually going to lead to the kind of regulation and certification other engineers are subject to.
Which means, while the safety is much better than raw C, with proper strings and arrays, alongside stronger type system, it still has UAF as possible gotcha.
It ought to be fairly straightforward. Zig is an easy dependency to either vendor or install on a given system/code-base (much more painful currently if you want Blaze to also build Zig itself), and at a bare minimum you could just add BUILD steps for each of the artifacts define in build.zig.
Things get more interesting if you want to take advantage of Zig's caching, especially once incremental compilation is fully released. It's a fast enough compilation step that perhaps you could totally ignore Zig's caching for now and wait to see how that feature shapes up before making any hard decisions, but my spidey senses say that'll be a nontrivial amount of work for _somebody_ to integrate those two ideas.
Also, with four space indentation (or for Go, a four space tabset), 'func' aligns right where the code begins, pushing the function name off one space to the right. For 'fn' the function name starts one space before the code, I find this more aesthetic. Then again, the standard tabset is eight spaces, so this matters less in Go.
It would be pretty silly to pick a language on the basis of that kind of superficial window dressing, of course. But I know which one I prefer.
can is the operative word, not is. Thinking of that other word, the m-word, that sounds like nomad. ;)
/jk
I literally just signed up to ask if anybody can recommend any good Zig codebases to read other than Tigerbeatle. How's your terminal going?
Edit: The rest of the posted site seems like a treasure trove not just this one article. Was wondering how to get into Zig and here we are. Such kismet.
Almost missed it so heads up for others.
If you're looking for interesting Zig codebases to read, you might be interested in our low-level audio input/output library[1] or our module system[2] codebase - the latter includes an entity component system and uses Zig's comptime to a great degree to enable some interesting flexibility (dependency injection, global view of the world, etc.) while maintaining a great amount of type safety in an otherwise dynamic system.
Rust was still probably right to not expose that at first since memory is supposed be fairly transparent, but Zig forces the user to care about memory, and given that constraint it's nearly free to also inform them of problems. The stdlib is already designed (like in Rust) around allocations succeeding, since those errors are just passed to the caller, but Zig can immediately start collecting data about how people use those capabilities. At a language level, including visibility into allocation failures was IMO a good idea.
In programs that don't have stack overflows, it's nice to be able to handle allocation failures though :)
Blocking (or rather, non-blocking, which is clearly what you meant) IO is a different story. Zig had an async system, but it had problems and got removed a couple point releases ago. There's libxev[0] for evented programs, from Mitchell Hashimoto. It's not mature yet but it offers a good solution to single-threaded concurrency and non-blocking IO.
I don't think Zig is the best choice for multithreaded programs, however, unless they're carefully engineered to share little to no memory (using message passing, for instance). You'd have to take care of locking and atomic ops manually, and unlike memory bugs, Zig doesn't have a lot of built-in support for catching problems with that.
A language with manual memory allocation isn't going to be the language of choice for writing web servers, for pretty obvious reasons. But for an application like squeezing the best performance out of a resource-constrained environment, the tradeoffs start to make sense.
I am working on an actor language kernel, and was thinking of adopting the same strategy, i.e. using a very naive bump allocator per actor, with the idea that many actors die pretty quickly so you don't have to pay for the cost of GC most of the time. You can run the GC after a certain threshold of memory usage.
Assuming the number of concurrent requests is known and they have bounded memory usage (the latter is application-dependant, the former can be emulated by 503-erroring excess requests, or something trickier if clients handle that poorly), yeah, just splay a bunch of bump allocators evenly throughout RAM, and don't worry about the details. It's not much faster though. The steady state for reset-arenas is that they're all right-sized contiguous bump allocators. Using that strategy, arenas are a negligible contribution to the costs of a 200k QPS/core service.
Since nothing can be writing to it during that anyway, I'm not sure the language level immutability makes a lot of difference to GC itself.
Very few were dealing with raw memory management in C, without anything else.
And all of this evolved into the Web application servers, and distributed computing landscape of modern times.