This reminds me very much of transputers. The idea here is that each cpu can context switch extremely quickly with minimal latency to any resource and you have a topology that is great for matrix maths as a result.
Your core needs to be fully programmable so you can do things like kernel fusion. The simplest form is to load quantized weights and dequantize them to bfloat16 as you go. Llama.cpp and it's gguf files support various types of quantization and most of them require programmability to efficiently support them.