Soft Machines VISC architecture

The other interesting thing is that the NVidia Denver 64 bits core is also supposed to do some dynamic code optimization to map ARMv8 machine code to an optimized representation running over a very wide implementation. I'm sure the details are very different, but from a very high point of view it's two companies claiming to extract higher IPC and performance/power efficiency with some form of code translation running on top of a very wide underlying implementation.

And in a slightly different way, Apple CPUs seem to be very wide and have a very good efficiency (no talk about dynamic code adaptation though).

Maybe there's something cooking, with different people independently coming from different angles toward some new rough optimum? We'll see. I'm more interested in Denver right now, we'll get the Nexus 9 soon with real benchmarks to see the actual potential.

The parallelism is there in most of the algorithms, and it gets taken out by a human translating the algorithm (which is inherently parallel) into some arbitrary language that the computer understands (most of which aren't parallel).

If you do a REALLY good job of pattern recognition coupled with some execution in a simulator you might be able to figure out what the core of a program is and go to work on that.

I'm skeptical of anything that reverses arbitrary machine code from serial to parallel but that's not to say it can't happen.

I did some work patching binaries on "basic blocks" which are chunks of code between branches. Inside a basic block you can reorder instructions however you'd like so long as you don't step on your own feet. The key piece of insight that enables this is that as long as the program state is correct by the time you get to a branch, it doesn't matter the EXACT order in which it got to the correct state.

If they've had some kind of bigger insight, it's not unreasonable to come up with short-timescale virtual threads. Cache latencies are on the order of a few to say a hundred cycles. Core to core latency should be about the same if you design it well. Having an insight that's bigger than basic blocks and scales up to say maybe 1000 cycles would enable you to shuffle computation from one core to another and back in time to make an improvement.

CPUs which do out-of-order execution already have a commit engine which either commits or throws away the results of speculative computation as a result of the branch predictor. To make something which operates at a larger (though not program-wide) scale isn't impossible to believe.

I can see it.

I mean, just multithreading embarrassingly parallel loops could often offer drastic speedups - the reason it's not generally done at the compiler level is due to overhead problems (by the time you're done creating threads / running them / cleaning them up / doing context switches and memory stalls you might as well have done the problem the straightforward way). If you're doing things at the hardware level you can get substantially less overhead.

In other words, don't think of it as "this can always multithread" - think of it along the lines of "this can pick up on common programming idioms and emit corresponding SIMD instructions".

On a related note: this sort of thing is one of the reasons I wish that instead of SSE we had something along the lines of a forall {chunk of code} instruction, (with the limitations that the code is executed in an unspecified order, and that the code cannot branch). Explicitly hardcoding the number of parallel data streams is... shortsighted.