This has probably been posted before. As interest in hardware acceleration for AI intensifies, I started to think about whether analog circuits would make more sense. You can imagine actually modeling neural network architecture in the circuits and using circuit response functions as the activations. In theory you could make this super fast.
I'm sure people smarter and more knowledgeable than me have thought this through...