Agile Design and Implementation of a Systolic Array-Based CNN Accelerator (2024) [pdf] | Hacker News Reader

Agile Design and Implementation of a Systolic Array-Based CNN Accelerator (2024) [pdf]

ijerm.com

Let's understand the whyness of the rather cryptic sounding Paper, above!

First understanding: LLMs/AI's require Matrix math (aka "matmul"), lots of it, billions, trillions (and perhaps more in the future!) times per second.

Second understanding: It would be nice if we could create our own Open Source hardware for this, via FPGA's. Oh sure using an FPGA probably won't be as fast as using a dedicated silicon ASIC -- but they're great for learning and prototyping such ASIC's.

Third understanding (from the following web page: https://www.alphamatch.ai/blog/cpu-gpu-tpu-npu-lpu-dpu-ai-ch...):

>"Google's TPU takes specialization one step further. At its core is a systolic array — a grid of multiply-accumulate (MAC) units where data flows through in a wave pattern. Weights enter from one side, activations from the other, and partial results propagate without returning to memory each time — eliminating the memory bottlenecks that plague GPUs.

The entire execution is compiler-controlled, not hardware-scheduled, making it extremely predictable and efficient. TPUs scale massively — a single TPU pod can contain up to 9,216 TPUs working in lockstep."

Or phrased another way (Einstein's dictum, "As simple as possible but not simpler"!), by getting rid of any unnecessary digital logic components not directly related to the core task of Matrix math, the design can be radically simplified.

Fourth understanding: SpinalHDL apparently offers a lot (over Verilog, VHDL, etc.) to simplify the design of repeated-digital-logic-component hierarchies, such as the systolic arrays / grid of multiply-accumulate (MAC) units used for creating a Matrix-math only processor, aka an "AI accelerator"...

Now, with all of those understandings firmly in place, you are now prepared to read the Paper, above... and/or the one below! :-)