I'd expect it to require too much RAM bandwidth to be feasible.
RAM is really slow at silicon speeds. Very little is reachable in one clock cycle, unless the clock cycle is abysmally slow.
RAM is really slow at silicon speeds. Very little is reachable in one clock cycle, unless the clock cycle is abysmally slow.
Example. If you have a neuron with 16 inputs each 8 bit wide and with a 4 bit weight per input, you will have 16 specialized multipliers each scaling its input by the corresponding weight and then the 16 scaled inputs feed into an adder tree and finally an activation function.
Then there's the current speedup in inference from restricting which subset of the model is used, which is not a "swap in" that would work with hard wired neurons.
But I dunno. Maybe. I'm just guessing.