Hacker News Reader

Comment by jacquesm | Hacker News Reader

There is something interesting happening here. For the longest time we've been doing everything we can to efficiently share a CPU with a low number of cores across a large number of threads. But as the number of cores increases we have the opposite problem: how to efficiently make a number of cores cooperate on the same task. Once the number of cores meets or exceeds the number of threads you can do away with a lot of the tricky bits that make our machines messy and insecure, a CPU per thread is so much cleaner. Especially if you can throw in some more isolation. And once you go down that road giving each CPU its own dedicated on-die memory is a relatively small step and you end up with a cluster-on-a-die that scales up and down from a power consumption point of view with the requirements because you can simply power off those chunks of CPU/RAM that you are not currently using.

Sort of like the ideal machine for the Qubes OS.

View on HN

There is no such thing as on-die memory. DRAM is made by a completely different semiconductor process and cannot be integrated with large-scale logic. The closest thing to memory-on-die is/was eDRAM which scaled up to at most tens of MB per chip, but at 7nm even IBM has abandoned eDRAM and gone back to SRAM.

I worked at an ASIC foundry in the late 1990's. We had an embedded DRAM option but it was basically the DRAM process with standard cells ported to the DRAM process. We only had a couple of customers ever choose it because the logic / standard cells were much slower in the DRAM process compared to the normal ASIC process.

Not DRAM but SRAM would be possible and there are many processors with on-die L3.

If you only need ~1MB per core, L2 already gives you that.

This problem has been solved - albeit sub-optimally for general purpose computing.

GPUs are massively parallel processors with 1000s of cores. While CUDA isn’t the easiest to work with, some Python libraries such as JAX and Tai-chi are attemting to remove the bar between CPU and GPU computing completely.

A program using either cab transparently switch between a CPU or GPU backend.

GPUs do not have thousands of physical cores, though. What's called a "CUDA core" is essentially a programming slash marketing abstraction for a SIMD lane. A closer analogy to a CPU core would be a Streaming Multiprocessor (RTX 4090, for instance, has 128 of them). But that comparison is still moot, because GPUs are simply not designed for executing branchy scalar code. They'd be laughably slow at it, so outside of easily vectorizable code that already takes advantage of SIMD instructions, I don't see how you could offload any CPU tasks to a GPU.

I think the major architectural leap would be MIMD with a standard ISA.

I think something along the lines of a Pentium Pro or an ARM core. Pentium Pro had 5.5 million transistors. A modern CPU has about 1000x more, so about a thousand Pentium Pro-grade processors would fit in die like a modern 7770X.

I'd take that over my GPU any day.

The hard and expensive part is, obviously, memory, cache, and interconnect. The even harder part is software. And the less hard part I'm intentionally oversimplifying is power consumption.

https://en.wikipedia.org/wiki/Multiple_instruction,_multiple...

An example of MIMD system is Intel Xeon Phi, descended from Larrabee microarchitecture.

https://en.wikipedia.org/wiki/Larrabee_(microarchitecture)

Its x86 cores were based on the much simpler P54C Pentium

https://en.wikipedia.org/wiki/Xeon_Phi

That is the core from the generation before the Pentium Pro. Larrabee was supposed to be a GPU, wasn't good enough to compete at that, then they rebranded it as Xeon Phi but cancelled it a few years ago.

I tried buying those but they wouldn't let me.

Well, FWIW xeon phis are about $40 on ebay now. Theyre obviously not much more than toy's now.

I'd love to see solutions like that. But what you are talking about is probably a niche market.

I don't think so. I think the core problem are network effects and momentum.

If someone could wave a magic wand, and there were OS, app, compiler, video game, etc. support for both MIMD and current architectures, I think MIMD would take over overnight.

Most of what computers do is ridiculously parallel. From each browser tab getting an isolated CPU, to having a spreadsheet spread out among cores, to rendering fonts in a document.

However, given a universe with trillions of dollars invested in the status quo, a disruption would need some sort of rather complex pathway, with some niche markets, some growth strategy, etc. As someone pointed out, Intel tried with Phi and failed.

I think the big driver could be security.

This is a shot in the wrong direction. GPUs are accelerators for code that, ultimately is run by CPUs. They might be headed to be more all-purpose computing hardware, but they aren't there yet, most importantly, from OS perspective.

To elaborate on this: OS creates processes, assigns ids to them, assigns other physical resources to them s.a. association with namespaces (which later gives them user permissions, network access, virtual memory access, filesystem access etc.) and then these processes are associated with some GPU resource.

If and when OS will start creating processes entirely on GPU, then it will make sense to talk about how GPUs are solving the problems of threads per core etc. For now it's a moot point.

This is not said to discourage though. I really feel like CPU-centric model of what we call "computers" is not a good one going forward. The "periphery" is growing smarter with each generation, and wants to do its own computing, and spread its load somehow, and we keep coming up with ad hoc solutions that don't mix well with CPU-based concurrency, s.a. async I/O or CUDA. We really need a different concept of concurrency that would be more uniform and at the same time more flexible across different devices that can do work concurrently, and this, interface if you will, must come from the operating system, not as a user-space library to be truly effective.

While there is a trend towards GPGPU this isn't there at the level where you can just go out and buy it like the dev kit in TFA. GPU compute capacity is still limited to a subset of all the use cases that you can use an ordinary CPU for and even though there is a chance that in the future the two will converge for now they are far enough from each other that I don't see the gain in the same sense that a large number of 'regular' cores would.

The fact that programs can switch between those two execution contexts seamlessly is a nice benefit but still isn't the same as having many ordinary CPUs. I've used GPUs extensively and I'm really happy to see their capabilities more integrated in everyday languages but they are at best for now a co-processor like device, a chunk of hardware that you offload specific parts of your workload (typically: the numerical chunk of it that is massively parallel).

>because you can simply power off those chunks of CPU/RAM that you are not currently using.

That is how IBM has handled their mainframes for quite a long time. You can hot-plug them in/out.

That is what a server processor is. A Beowulf cluster on a die.

Not quite. Beowulf clusters tend to use regular networking interfaces for IPC (and sometimes more specialized ones). This would not even necessarily have to be one die, that doesn't matter so much as long as it gets packaged as a single SKU. All of the interconnect would be the same as it is on this 80 core sample here. Typically I have about 400 processes running on a machine with several hundred more threads. That's within striking distance of this kind of device. Just a few more iterations and we'll be there, and a multi socket solution would already get very close. The next step would be to integrate the memory.

Ampere Altra 128 core telco edge server benchmarks = 100 RPi4 Beowulf cluster and 22% more energy efficient. https://youtu.be/UT5UbSJOyog

Reply on news.ycombinator.com