Comment by wildmusings | Hacker News Reader

Comment by wildmusings | Hacker News Reader

If you're routinely zeroing this much memory and the performance matters, you might benefit from idle-zeroing it. That is, when you need to zero the massive block, just switch to a different block that has already been zeroed or partly-zeroed in the background. Whatever hasn't already been zeroed, finish synchronously. The background thread doing the zeroing would be scheduled with the lowest priority, so that it only runs when the system otherwise has nothing to do.

At first, I thought you might just want to get fresh pages from the kernel (which are always zeroed), but this answer convinced me that might not actually be faster because of the overhead from syscalls and fiddling with virtual memory https://stackoverflow.com/questions/49896578/fastest-way-to-... . And Linux doesn't idle-zero or pre-zero pages (though I believe there's a switch to enable pre-zeroing for purposes of security hardening), so you're probably gonna end up with the OS doing a synchronous[1] zeroing anyway.

[1] Synchronous from when you actually first write to each page. My understanding is that when your process gets new pages, they're all mapped to a special zero-page and set to copy-on-write. So there is still some efficiency here in theory: you don't have a long wait for the entire range to be zeroed all at once and you never have to zero pages that you don't modify.

Have you tried this and measured it? It feels like the overhead of having to use synchronization might eat up all your savings.

It's an interesting idea, mostly applicable to long running processes (e.g,. a server) where the concept of "idle time" applies.

For something like a process that starts up, does its job as quickly as possible and then stops, it is less obvious how to do it: you could certainly start another thread to do this, but if you are willing to add threading you are probably better off just trying to parallelize the underlying work, rather than doing zero async.

The main downsides to this approach:

1) It is cache unfriendly: if you zero the pages synchronously immediately prior to use, you bring them into cache during the zeroing and then they are hot when you use them on the same thread. For idle zeroing this link is lost so you are likely to bring them into cache twice: once during the zeroing and once during the subsequent use.

2) On NUMA systems, doing the idling on some thread/CPU distinct from where you will use it is an anti-pattern: with the usual "first touch" allocation, the memory will be allocated on the node doing the zeroing, potentially different from the node doing the using - which could be an arbitrarily large penalty over time (as you keep using the remote memory for an arbitrarily long time)

Note that (1) is a common confounding factor when measuring the cost of zeroing. E.g., if you have something like:

    zero(mem)
    do_work(mem)

You might measure the zeroing taking 500 (time units) and the work taking 800, total of 1300. Lets say you find a clever way to totally avoid the zeroing (e.g., maybe do_work didn't actually need zeroed memory): you expect only the 800 time units of work to remain - but actually measure 1200: the zeroing more than just zeroing: it was bringing in all the memory into cache and suffering all the cache, TLB misses, etc. When you remove it, that work just moves over to the subsequent work.

In some cases do_work alone can even be slower than the sum of zeroing and do_work (e.g, 1,500 time units) - because the interleaving of the work with cache misses can be less efficient in an MLP sense (because the work clogs up the OoO buffers so fewer memory requests fit in the window).

I don't think it's about long-running vs. short-running. That might be a heuristic, but what really matters is whether there is idle CPU time to be had between when you need large zeroed chunks of memory, or even if not, if you have some other performance characteristic you want (e.g. stable, consistent throughput) that would benefit from parallelizing the zeroing instead of having to periodically block.

Yes, it was just a terse way of putting it: by long running I really mean something that runs more or less indefinitely, usually serving "external" requests, like a server or daemon vs "short running" where I mean something that is triggered to do a specific job: think a compiler or compressor or whatever.

Of course you could have short running servers or long running full-CPU processes too.

Long/short probably are the

Is that how languages that always give you zeroed arrays (e.g., java) do it?

There is no canonical answer to that question. Compilers wary wildly in implementation details.

I doubt it. Needing to zero ultra large chunks of memory often enough that you care about performance to this extent is a very niche scenario. If I had to guess, Java probably just zeroes the memory upon garbage collecting it or before handing it back out; fresh pages from the OS are already zeroed.

But anything is possible. Maybe some big customer used this pattern and now Java detects it and does something fancy like what I suggested. But really, this is an unusual scenario for an application.

There's a brilliantly-named paper on this: Why Nothing Matters: The Impact of Zeroing.

Efficient zeroing is something JVMs put some thought into. Well, with a notable exception, in HotSpot itself: We were able to substantially improve its performance by using memset() to perform the zeroing.

[0] (PDF) http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.225...

In that case, why not request freshly zeroed pages from the operating system?

Because if you're needing to repeatedly free giant amounts of RAM, the OS probably doesn't have many zeroed pages and will need to zero them itself. Linux, at least, does not idle-zero, so if you want to take advantage of otherwise idle CPU time to do the zeroing, then you need to take matters into your own hands.

Reply on news.ycombinator.com