That is the big "if", to be sure. I spent three years working for Intel after they acquired Vertex.AI, trying to help build that software story, and it was rough going; we didn't accomplish much, and it doesn't sound like they're using any of it.
I was impressed by their _CPU_ kernels in OpenVINO which I studied very closely after discovering I cannot beat their performance even on kernels specialized for my specific tensor shapes. Turns out the reason I can't beat them is because they jitted their kernels with xbyak, and specialized them for those shapes, too. They also obviously knew exactly how to maximize the ALU utilization, and counted every cycle, something that's very hard to do unless you specialize _just_ on CPU performance and have access to the internal documentation. Top notch work. But that team used to be in Russia back then, in Nizhny Novgorod, IDK if it's still a part of Intel.
Yes the _CPU_ perf of OpenVINO is a thing of beauty. Debuggability is a nightmare story but if it works you're GOLD. The GPU/Myriad story was very clunky last times I tried.
I hope some Intel PHB reads this and purely stochastically makes the right decision for the first time in his/her life. It seems pretty existential to be competitive at selling shovels during a gold rush.
Looking at OpenVINO repo activity, seems that most of them got relocated and still work at Intel.
There's a Matt Pharr-level story to tell here if you feel like it one day. I'm also looking at what the very cool maxas authors became. Intel seems like a dark hole of blackness and holeness for acquired tech these days.
1024 fp16 macs is pretty good but the 128b vector datapath is weak sauce. on the other hand 2MB SRAM is legit. i wonder how many tiles (i don't think it's in the post).
I think the main advantage of Movidius was its memory architecture. They had to disclose the HW arch for some sort of a tender in the EU which is how I stumbled upon their HW documentation years ago. Basically IIRC even though you’re right and 128 bit is weak sauce, the strength of that arch was that the memory was much “closer” to the cores and it was partitioned and accessed by multiple cores at the same time, boosting overall available bandwidth. The weakness of it was that it was (IMO) overly complicated for no good reason, and imposed a rather inflexible programmability model which if their software layer didn’t do certain things, you couldn’t do them at all. Which was a problem before transformers, because models tended to use a greater variety of ops