Intel Meteor Lake's NPU

Indeed, silicon design is long in its NPM left-pad phase. Which begs the question why it is still so open-source averse and patent-pilled.

Pretty sure ME is Intel Quark based now. Quark is intel's weird 32 bit x86 as a microcontroller chip.

It must be a bit depressing for Intel engineers that they are forced to keep themselves tied to the ever extending x86 ISA and can't explore all the other ISAs they use (or invented). The 860/960 generated a lot of buzz back then and there was some pretty capable hardware using it.

Fortunately, with open-source becoming dominant in the data center and the relative ease of bootstrapping a new ISA on top of it, we will see some extra diversity in this space, but I don't think Intel will abandon x86 anytime soon (and "soon" is in almost geological scales here).

Didn't some keyboards and mice use embedded Z80 or 6502 CPU's?

Isn’t a big advantage of an accelerator is that its more optimized to do this sort of work than a cpu? This npu is a tiny dot on the Meteor Lake die while the cpus take up a whole tile and accounts for much of the power usage of the soc at any given time.

It makes systems "feel" way more responsive. It's a huge quality of life increase. Apple Silicon's user/product story has basically changed how people view this stuff forever.

4 TFLOPs is significantly faster than your CPU. Even 1.35 TFLOPS represents a reasonable edge. The question should be why Intel added another processor instead of extending AVX.

I was impressed by their _CPU_ kernels in OpenVINO which I studied very closely after discovering I cannot beat their performance even on kernels specialized for my specific tensor shapes. Turns out the reason I can't beat them is because they jitted their kernels with xbyak, and specialized them for those shapes, too. They also obviously knew exactly how to maximize the ALU utilization, and counted every cycle, something that's very hard to do unless you specialize _just_ on CPU performance and have access to the internal documentation. Top notch work. But that team used to be in Russia back then, in Nizhny Novgorod, IDK if it's still a part of Intel.

There's a Matt Pharr-level story to tell here if you feel like it one day. I'm also looking at what the very cool maxas authors became. Intel seems like a dark hole of blackness and holeness for acquired tech these days.

I think the main advantage of Movidius was its memory architecture. They had to disclose the HW arch for some sort of a tender in the EU which is how I stumbled upon their HW documentation years ago. Basically IIRC even though you’re right and 128 bit is weak sauce, the strength of that arch was that the memory was much “closer” to the cores and it was partitioned and accessed by multiple cores at the same time, boosting overall available bandwidth. The weakness of it was that it was (IMO) overly complicated for no good reason, and imposed a rather inflexible programmability model which if their software layer didn’t do certain things, you couldn’t do them at all. Which was a problem before transformers, because models tended to use a greater variety of ops