Hacker News Reader

Comment by jrmg | Hacker News Reader

Trimming the working set of a process doesn’t actually save memory. It just moves the memory from the working set of the process to the standby list. Then, if the system is under memory pressure the pages in the standby list are eligible to be compressed, or discarded (if unmodified and backed by a file), or written to the page file. [… If] the system has gobs of free and available memory then it may never do anything with the page, making the trimming pointless. The memory isn’t “saved”, it’s just moved from one list to another. It’s the digital equivalent of paper shuffling.

This sounds like it should be basically free in a memory-unconstrained system - is the system really just spending all its time managing these lists? Why is it so expensive?

View on HN

When the address isn't in the process page table accessing it traps into the kernel; then you've got to do all the cache clearing stuff for speculative execution vulnerabilities; then the paging logic needs to figure out what to do, in this case because of the set limit, you've also got to demote at least one page, which means updating the page table, but also sending IPIs to all processors to flush that page/those pages from the TLBs; and then you return to execution.

If you're accessing a lot of pages, mostly randomly, it's going to be a mess. If you're decompressing files (as an installer might do), and the compression window is large relative to the limit (as it might be to get the highest compression ratio), that could be a lot of mostly random access to pages, causing a lot of page faults, and high cpu.

Also context switching between kernel and user mode is now heavier than ever courtesy of speculative execution mitigations.

I don't know about Windows, but the Linux kernel is entirely fine to just keep rarely used memory pages around as long as it doesn't need the memory. If you don't need the memory otherwise, having the page loaded and cached is better than needing to load it.

Even though, then you end up with confused reports about "Linux eating all the memory" and "an idle system using all the memory (for caching)".

AFAIK, newer kernels also have a preemptive swap-out of idle pages so they can evict the pages quicker, so an idle system might even be swapping (out). This results in even more confusion.

My impression is that Windows is somewhat aggressive at paging or using memory compression (as a faster alternative to a write to swap), with the goal of freeing more space for cache use. Not very aggressive, but a lot more than Linux.

The effect of caching on Windows can be quite pronounced if you process large datasets on a machine with a large amount of RAM.

If, say, you have 256GB RAM and a 100GB folder of ~1GB files, you will only ever have a few GB used actively. A first pass of processing over the folder will take a long time (reading from disk). Subsequent passes will be much faster, though, because reading is done from the RAM-cached versions of the files (the output from the previous run, was my understanding).

> "Linux eating all the memory" and "an idle system using all the memory (for caching)".

I find these reports bewildering (alongside CPU usage). You would hope that something would be exploiting your expensive machine to the fullest (so long as the work being done is useful/intentional/desired - obviously not wasteful page thrashing).

There's the power consumption issue, of course.

For CPU time, yes, but I don't think DRAM power consumption for idle refreshing varies meaningfully depending on the data stored in it.

You don’t see it in x86 I think, but I believe ARM mobile SoCs can completely turn off banks that are unused. I haven’t followed to see if anyone actually does this yet, but I recall hearing about attempts to turn off unused RAM when in sleep (when awake, the RAM refresh cost is negligible). To turn off the RAM implies that you might want to prefer to place cache data in the banks that you will want to turn off so that you can just drop caches on sleep quickly. Of course, whether that’s actually beneficial in practice is hard to say - some caching you want to be persistent as you’ll need it on wake anyway.

Outside cases where you want to optimize for sleep performance, you probably don’t want to do this though because NUMA systems have conflicting requirements for storage (but thankfully for now the systems with NUMA and the systems that benefit from turning off RAM refresh when sleeping has 0 overlap). Consumer desktops are probably not worth doing this on and laptops probably isn’t a huge difference because of battery size / usage patterns.

You're probably thinking of DDR self-refresh which lets the entire CPU (including the memory controller) power off without losing data when the device is in sleep mode. I've never seen hardware that partially powers off memory the way you described.

My memory could be faulty, but I didn’t think it was self refresh which has been quite common for a long time. Maybe it’s possible the OS vendors never got the idea working on the software side and abandoned it / in practice you could never arrange to have a good likelihood of having an entire bank of memory unused. You could estimate the benefit with SW experiments cheaply before ever going down the path of building the required HW support.

But it's a pretty cool idea :-)

https://youtu.be/beefUhRH5lU

Yep, that is generally what Windows does, using available memory for the disk cache. In the user trace the system had lots of disk cache (available memory) and lots of _free_ memory, so there really was zero value in trimming the working set.

But, (undocumented) rules gotta be followed, so trim the working set it is.

Reply on news.ycombinator.com