Ascenium wants to reinvent the CPU and kill instruction sets altogether

> The idea of a compiler-based software solution embedded in an architecture would theoretically allow the Aptos processor to interpret workload instructions and distribute them across processing resources in such a way that the amount of work being parallelised is as close as possible to the theoretical maximum, whilst taking advantage of having much lesser architectural inefficiencies than instruction-based processors.

Whether there will be an instruction set, or not, you will have to have some kind of convention for the bitstream for the re-programmable logic, and very likely you will re-implement a kind of an instruction to do that with high level of efficiency.

Modern CPUs already all run microcode on some kinds of a programmable fabric in the front-end to give some degree of programmability for how the front-end, and back-end interact.

That's why the target for this sort of technology is GPGPU work. The streams of data are very regular and largely a series of batch jobs.

For interactive systems with millions of context switches and branches all the "Sufficiently Advanced Compilers" all fall down. There's just not enough commonality between consecutive operations for ahead of time optimizations to occur. Hardware that does great in batch jobs ends up suboptimal for the insanity that is interactive code.

I bet people on this forum know why that is / isn't true and I am curious.

> Extracting enough fine grain parallelism is hard. Manually creating it is tedious and difficult.

A big win for NVidia was tricking enough programmers into manually describing large swaths of parallelism available for a GPU to take advantage of.

Manually creating fine grained parallelism through classical structures (mutex, semaphores, async, etc. etc.) is tedious and probably counterproductive. But it seems like describing __some__ extremely common forms of parallelism is in fact very useful in certain applications (matrix multiplications).

OpenMP, CUDA, OpenCL, (and even Fortran/Matlab to some extent)... even Tensorflow... are showing easy ways to describe a parallel problem.

CPUs seem to be the best architecture at discovering latent parallelism in general purpose code that the original programmer wasn't aware of. (Typical programmers are not aware of the multiple-pipelines that a modern CPU tries to fill up... and yet that code continues to accelerate from generation-to-generation... now with Apple's M1 proving that even 8-way decode is reasonable and provides performance gains). CPU designers seem to be experts at finding fine-grain parallelism as it is, and I don't think anyone can beat them at it. (Only another CPU designer will probably make something better).

---------

But for these alternative compute paradigms: such as GPU-compute, there are far fewer programmers and compiler experts working on that architecture. Its far easier to find little gold-nuggets here and there that lead to improved execution.

SIMD-languages such as C-Star, Lisp-Star, CUDA, OpenCL, OpenMP... have been describing parallel patterns for decades now. And yet, it only feels like we've begun to scratch that surface.

I used to believe that FPGAs could compete with GPUs in throughput compute cases thanks to high-level synthesis and OpenCL, but it looks like most applications have gone with something more ASIC-like than general-purpose (eg TPUs and matrix units in GPUs for ML).

You can get a commodity CPU today at 4.7+ GHz clock and 64 MB of L3 cache (AMD Ryzen 9 5950X). There's no FPGA / configurable logic in the world that comes even close. AMD's even shown off a test chip with 2x96 MB of L3 cache.

GPUs __ARE__ an ASIC, but with a different configuration than CPUs. An AMD MI100 comes with 120 compute units, each CU has 4x4x16 SIMD-lanes and each lane has 256 x 32-bit __registers__. (AMD runs each lane once-per-4 clock ticks)

That's 32MB of __registers__, accessible in once every 4th clock tick. Sure, a clock of 1.5GHz is much slower but you ain't getting this number of registers or SRAM on any FPGA.

> I am not an expert in this space, but it seems like the only algorithms I've seen lately people proposing custom hardware for at scale are neural network evaluation

The FPGA / ASIC stuff is almost always "pipelined systolic array". The systolic array is even more parallel than a GPU, and requires data to move "exactly" as planned. The idea is that instead of moving data to / from central registers, you move data across the systolic array to each area that needs it.

----------

CPUs: Today's CPUs __discover__ parallelism in standard assembly code (aka: Instruction level parallelism). It spends a lot of energy "decoding", which is really "scheduling" which instruction should run in which of some 8 to 16 pipelines (depending on AMD Zen vs Skylake vs Apple M1). Each pipeline has different characteristics (this one can add / multiply, but another one can add / multiply / divide). Sometimes some instructions take up the whole pipeline (divide), other times, multiply takes 5 clock ticks but can "take an instruction" every clock tick.

CPUs spend a huge amount of effort discovering the optimal, parallel, execution for any chunk of assembly code, and today is scanning ~400 to 700 instructions to do so (the rough size of a typical CPU's reorder buffer). That's why branch prediction is so important.

In effect: CPUs are today's ultimate MIMD (multiple instructions / multiple data) computers... with a big decoder in front ensuring the various pipelines remain full.

-----------

GPUs: SIMD execution. If parallelism is known ahead of time, this is a better architecture. Instead of spending so much space and energy on discovering parallelism, you have the programmer explicitly write a parallel program from the start.

GPUs are the today's ultimate SIMD computers (single instruction / multiple data) computers.

-------------

Systolic Array: Will never be a general processor, must be designed for each specific task at hand. Can only work if all data-movements are set in stone ahead of time (such as matrix multiplication). Under these conditions, is even more parallel and efficient than a GPU.

When Intel shipped a compiler that artificially, and purposefully, crippled performance of resulting binaries when run on an AMD CPU.

Not what you had in mind, but that dirty trick compiler did give Intel an advantage.

Not that GPUs and their proprietary fragmented & buggy tooling are that nice for developers even now, 15-20 years into the attempt, and the vast majority of apps still don't bother with it. And of course the whole GPGPU thing was just riding on the wing of gaming for most of its existence so had a really long artificial runway.

- ILLIAC 4 (64-cpu mainframe, 1970s): "A matrix multiply is a master's thesis, a matrix inversion is a PhD thesis, and a compiler may be beyond the power of the human mind".

- Connection Machine. (SIMD in lockstep, which just wasn't that useful.)

- NCube (I tried using one of those, 64 CPUs in an array, each with local memory, message passing hardware. It was donated to Stanford because some oil company couldn't find a use for it. Someone got a chess program going, which works as a distributed search problem.

- The Cell CPU in the Playstation. (Not enough memory per CPU to do much locally, and slow access to main memory. Tied up the entire staff of Sony Computer Entertainment America for years trying to figure out a way to make it useful.)

- Itanium. (I went to a talk once by the compiler group from HP. Optimal instruction ordering for the thing seemed to be NP-hard, without easy approximate solutions.)

Those are just the major ones that made it to production.

But then came GPUs, which do useful things with a lot of loosely coupled compute units on a single chip. GPUs have turned out to be useful for a reasonable range of compute-heavy tasks other than graphics. They're good for neural nets, which are a simple inner loop with massive parallelism and not too much data sharing. So there may now be a market in this space for architectures which failed at "general purpose computing".

The SASS assembly on NVidia's instruction set seems to be manually creating read/write barriers at the assembly program level instead of leaving it up to the decoder. PTX exists as the intermediate step before SASS for a reason: all of that read/write barrier placement is extremely complicated.

CUDA cheats by making the programmer do a significant amount of the heavy lifting: the programmer needs to write in an implicitly parallel style. But once written in that manner, the compiler / computer can execute the code in parallel. The biggest win for NVidia was convincing enough programmers to change paradigms and write code differently.