Hacker News Reader

Comment by whalesalad | Hacker News Reader

That is a great idea. I think the software is going to be the hard part. You would need some kind of heuristic or software to manage moving memory between those two locations. That is just my initial thought I could be totally wrong.

View on HN

Every modern general purpose computer already has multiple layers of memory. This would just be an additional layer. The virtual memory subsystems in the OS will handle this. At the end of the day it's just caches all the way down. A workstation with 16GiB of "on-chip" memory would be like a huge L4 cache for the say 512GiB of "standard" DDR4.

I really like OSTEP's chapters on virtual memory if you're interested in reading more[1].

[1] http://pages.cs.wisc.edu/~remzi/OSTEP/vm-complete.pdf

Thanks for the link, downloaded to read this evening :thumbsup:

Yet, is it a requirement in a memory hierarchy to copy the lower level in the upper one? Like, all the stuff in you L1 cache has to be also in your L2 cache. E.g. if you would have 32gb of external DDR it would add only 16gb more to the packaged 16gb?

It's not a requirement, and different systems in the past have made different choices.

https://en.wikipedia.org/wiki/Cache_inclusion_policy

It's definitely not, and in fact swap on a modern OS doesn't work that way. Something can be only in physical RAM, only in swap, or (when "clean"/unmodified vs the copy in swap) in both (allowing it to be dropped from physical RAM quickly). So one obvious approach would be to simply use the external RAM as swap.

Relatedly, I've heard of using Intel Optane as "slow RAM" for cold pages, and I think the idea there is also that it'd be in one or the other but not both. (Optane can be thought of as very expensive/fast flash or very slow/cheap RAM.)

Pretty sure this is already a thing for NUMA systems, e.g. an Intel system with a pair of N-core processors. Each processor gets "its own" half of memory. Memory which belongs to the other processor is slower to access.

See also Linux cpusets (cset command), which can be used to control which NUMA nodes a process has access to.

Luckily that's a problem that academics have been working on since the 1960s. https://en.wikipedia.org/wiki/Non-uniform_memory_access

You'd be surprised how many modern OSes are at least partially NUMA aware. Even Java already has a non-uniform aware allocator.

macos already has capabilities for compressing memory used by applications before finally giving up and paging it out. It seems plausible they could extend the code that supports that feature to push memory to external (or allocate it there in the first place if utilization isn't expected to be hot), and only pull it back after utilization indicates there is a benefit.

The same logic they already use for L1/L2 caching, and/or VM could pretty easily be adapted.

Maybe the memory controller could handle this, spilling onto the slower memory, much like swapping.

Reply on news.ycombinator.com