Hacker News Reader

Comment by jeffbee | Hacker News Reader

There are fundamental issues here and I think the article only touched on a few. On the software side this completely blows up the whole virtual memory concept. We will need different operating systems.

View on HN

Maybe PIM will push this forward, but I still think we're doing something fundamentally wrong by not just embracing NUMA and trying to do something Sun tried decades ago, which is have number of cores share 4GB of semiprivate working memory.

We've kind of half-assed it with DDR memory banks, but it mostly introduces mysterious slowdowns that are difficult to reason about and I think we would be better served I think by making a formal thing. Instead of introducing an L4 cache we could do this instead, and reduce the size of the L1-L3 caches, which shortens lookup time and thus latency.

For legacy apps, you could provide facilities for the OS to 'page' blocks in from main memory, but the speed would come from managing the workload imperatively, starting loads in the background before the data is actually needed, and dumps after it is last touched.

Intel SCC tried the extreme version of that. Though you could argue their Message Passing Buffers weren't shaped quite right (too small, hard for OS to use across trust domains). https://en.wikipedia.org/wiki/Single-chip_Cloud_Computer

You could also take a half-step in that direction with the DEC Alpha processor's extremely relaxed memory consistency model, and see how well things program for that. When I had a chance to poke at them, Alphas were about 2-3x faster than Intels, with much less ecosystem optimization effort burned -- and most software worked just fine.

why would it? the parent OS can already handle physically contiguous allocations so these should be no different (with the exception that a separate interface can be used to do compute over these buffers/pages).

If it requires physically contiguous RAM to work, then it's not really participating in the full virtual memory system, really. It would be using an exception to it, that can be accommodated to some extent by the OS, but not sitting in demand-paged storage like the rest of the system.

And the security folks will have tons of fun with this enormously leaky abstraction too.

Hey, just don't store your pagetables in PIM capable sections :)

Linux has been dealing with this kind of thing for over a decade now. Specialized SoCs love their memory carveouts.

You could still do map-reduce operations, but for it to really fly, what you'd want is a side channel between the memory chips that allows the reduce to happen out of band from the front-side bus and the reduction to be sent to the CPU. And any workflow where you can stream the reduction to the CPU that would be even better for latency.

That really doesn't make sense because your system is already using memory that needs to be physically contiguous but mapped virtually. Also, would you say pinned memory is not part of the virtual memory system??

You're only restricted by the fragmentation of the system memory which is an issue yes, but it's dealt with in other ways.

Reply on news.ycombinator.com