GPUs are massively parallel processors with 1000s of cores. While CUDA isn’t the easiest to work with, some Python libraries such as JAX and Tai-chi are attemting to remove the bar between CPU and GPU computing completely.
A program using either cab transparently switch between a CPU or GPU backend.
I think something along the lines of a Pentium Pro or an ARM core. Pentium Pro had 5.5 million transistors. A modern CPU has about 1000x more, so about a thousand Pentium Pro-grade processors would fit in die like a modern 7770X.
I'd take that over my GPU any day.
The hard and expensive part is, obviously, memory, cache, and interconnect. The even harder part is software. And the less hard part I'm intentionally oversimplifying is power consumption.
If someone could wave a magic wand, and there were OS, app, compiler, video game, etc. support for both MIMD and current architectures, I think MIMD would take over overnight.
Most of what computers do is ridiculously parallel. From each browser tab getting an isolated CPU, to having a spreadsheet spread out among cores, to rendering fonts in a document.
However, given a universe with trillions of dollars invested in the status quo, a disruption would need some sort of rather complex pathway, with some niche markets, some growth strategy, etc. As someone pointed out, Intel tried with Phi and failed.
An example of MIMD system is Intel Xeon Phi, descended from Larrabee microarchitecture.
https://en.wikipedia.org/wiki/Larrabee_(microarchitecture)
Its x86 cores were based on the much simpler P54C Pentium
https://en.wikipedia.org/wiki/Xeon_Phi
That is the core from the generation before the Pentium Pro. Larrabee was supposed to be a GPU, wasn't good enough to compete at that, then they rebranded it as Xeon Phi but cancelled it a few years ago.
To elaborate on this: OS creates processes, assigns ids to them, assigns other physical resources to them s.a. association with namespaces (which later gives them user permissions, network access, virtual memory access, filesystem access etc.) and then these processes are associated with some GPU resource.
If and when OS will start creating processes entirely on GPU, then it will make sense to talk about how GPUs are solving the problems of threads per core etc. For now it's a moot point.
This is not said to discourage though. I really feel like CPU-centric model of what we call "computers" is not a good one going forward. The "periphery" is growing smarter with each generation, and wants to do its own computing, and spread its load somehow, and we keep coming up with ad hoc solutions that don't mix well with CPU-based concurrency, s.a. async I/O or CUDA. We really need a different concept of concurrency that would be more uniform and at the same time more flexible across different devices that can do work concurrently, and this, interface if you will, must come from the operating system, not as a user-space library to be truly effective.
The fact that programs can switch between those two execution contexts seamlessly is a nice benefit but still isn't the same as having many ordinary CPUs. I've used GPUs extensively and I'm really happy to see their capabilities more integrated in everyday languages but they are at best for now a co-processor like device, a chunk of hardware that you offload specific parts of your workload (typically: the numerical chunk of it that is massively parallel).