Hacker News Reader

Comment by gpderetta | Hacker News Reader

What's the use of filling your ram with zeros when the data needs to be on L1, L2 or L3? Unless you are memsetting hundreds of MBs of memory, memset/memcpy in practice need to be handled by the cpu or something very close to it.

Zen has CLZERO which can clear a cacheline in one go, but not sure how good it is.

View on HN

This would be a CPU command that works with the RAM controller rather than something you control yourself (kernels to my knowledge don’t talk directly to the controller beyond maybe some basic power management, if that).

There is a definite need to do hundreds of MB - the Linux kernel has a background thread that does nothing but zero out pages. What do you think happens to the GBs of RAM freed by closing Chrome? Once it’s made available in one spot, no reason others could use it (eg a hardened malloc implementation, etc).

Interesting that you mention linux, because Linus has very very strong opinions about this :)

What is his opinion about this?

He strongly believes that something like rep stos/rep mov is the right interface for memset/memcpy and off-core accelerators (like DMA) are misguided.

His reasoning or rant for this?

I'm not sure about Linus's objections, but I've found that DMA accelerators for time sharing systems with general workloads haven't reaped benefits, as the overhead of multiplexing and synchronizing with them kills most of their benefits. At that point it's easier to blit memory yourself.

What are they?

If you find yourself doing this a lot, there's write combining memory to coalesce writes to be more friendly to the RAM controller.

Additionally, CLZERO ends up doing very similar work since the resulting cache flush os seen by the RAM controller as a block write.

Such ram capability would result in implementing hybrid compressed caches. Why waste whole cache line for storing zeroes when you can have dedicated compressed representation.

On a similar note part of ATI 2000 https://en.wikipedia.org/wiki/HyperZ was fast Z clear, today a norm on every GPU.

PPC has an instruction to 'load' a line ignoring its previous contents (just set up the cache state). useful in any case when you know you're going to overwrite the whole thing.

I used dcbz extensively back on the Wii.

>"Unless you are memsetting hundreds of MBs of memory"

Not hundreds but in one of my apps I do have 10th MB of continuous cache that has to be zeroed before use / reuse.

I wonder if someone was zeroing enough memory, where the memory is a private anonymous mapping, they might use madvise() with MADV_DONTNEED, which in linux will effectively zero the pages.

It returns the memory to the OS, and will pagefault on later accesses remapping them as zero-filled. It works in pages. Sizes smaller than a page result in whatever else is in the same page getting nuked.

If you don't immediately reuse the whole cache, it might spread out the zeroing/remapping over time, rather than in a single large go. Imagine some testing would be in order to see if a syscall + mapping changes ( require reloading with TLB for the process ? ) would be smaller than a straight run of writing zeros at some point.

IIRC, the zeroing is not something you can expect from non-linux madvise implementations.

Reply on news.ycombinator.com