How GPUs and AI are rooted in DSP and how the lack of generalized DSP chips (perhaps built from FPGAs) have set back progress in concurrent programming immeasurably, especially for the embarrassingly parallel problems whose solutions are making headlines today.
The point you're making about untapped technical potential is interesting, but I'm left wanting to know more about what you think the issue/solution is.
It would be so much better if we had generalized multicore computing. Something on the order of 256 or more 1 GHz MIPS, ARM, or even early PowerPC processors. We need to be able to run arbitrary code with flow control and be able to experiment with our own models for data locality, caching, etc.
Something like that running Erlang/Go/Octave/MATLAB or one of the many pure functional languages would open up amazing opportunities if we were no longer compute-bound. I first started caring about this in the late 90s with FPGAs but that possible future was supplanted by the hardware accelerated graphics future we're in now. Yes it's pretty good, but a pale shadow of what's possible.
Edit: here are a few examples of what I'm talking about:
https://news.ycombinator.com/item?id=15099422
The modern GPU is a “generalized multicore” machine anyways. Nvidia GPUs have, say, 60 SMs with many warps per SM concurrently resident, giving you hundreds of hardware threads (warps) available, each with 32-wide SIMD units. You have multiple memories you can play with for data locality (the multi-MB sized register file and shared memory space, the smaller caches, global memory). It’s a hard enough problem to use all of that well for all but simple problems.
If you want to change deeper behavior, there's always Verilog.
They do exist, it's just a considerable challenge to program them effectively and they're more expensive than GPUs. Whereas with GPUs the R&D is spread across the huge games market (and more recently cryptocurrency).
- https://news.ycombinator.com/item?id=13741505 shows an experimental new CPU architecture which appears to be single-core but demonstrates interesting power savings and speedups by eschewing standard on-chip caching and building a smarter compiler. TL;DR closed-source compiler, but I think the people making the design are trying to make the architecture design/registers/overview/etc as open as practical/viable with free documentation etc. I'm not sure if there's a simulator for it. I'm very sure they're already taking on commercial customers, and getting access to it is likely to be not overly nontrivial. (I realize you're not likely to need one just yet, but "can't access at all" makes for something much less interesting to file away.)
- Forth, Inc. makes the GA144, a 144-core microcontroller that natively runs Forth (and will NOT comfortably run transpiled <any other language here>). At one point a production company was making evaluation boards. https://www.youtube.com/watch?v=NK1zlz67MjU is a (1hr long, somewhat slow-paced) presentation that provides a good overview of the GA144. I half wonder whether I'm looking at a chip designed by people who don't understand SMP when I watch this, but I also think that it's possible that the brutal simplicity that Forth (and Chuck Moore) applies to everything means that I'm really staring at the raw complexity of SMP here. I've read that Forth is one of those languages that changes the way you think about programming, so maybe the GA144 could be a training processor - crack it, and you'll understand SMP a tiiiny bit better on more contemporary/traditional processor architectures.