We will see a completely new type of computer, says AI pioneer Geoff Hinton

With the mention of memristors, this sounds kind of similar to what Wolfram recently said about hardware for neural networks that can simultaneously be memory and compute:

> But even within the framework of existing neural nets there’s currently a crucial limitation: neural net training as it’s now done is fundamentally sequential, with the effects of each batch of examples being propagated back to update the weights. And indeed with current computer hardware—even taking into account GPUs—most of a neural net is “idle” most of the time during training, with just one part at a time being updated. And in a sense this is because our current computers tend to have memory that is separate from their CPUs (or GPUs). But in brains it’s presumably different—with every “memory element” (i.e. neuron) also being a potentially active computational element. And if we could set up our future computer hardware this way it might become possible to do training much more efficiently.

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

The article is about mortal computers which don't sound very interesting or special. I mean you can build analog computers today and do backpropagation on them and slap the mortal label on them as if it was some consumer brand. The "mortal computer" is a red herring.

What I find far more interesting is that he presumably had discovered an alternative to backpropagation.

I'm not sure exactly what Hinton means by "mortal" computers, but it sounds like he is embracing what Carson Mead was advocating with neuromorphic computing and analog VLSI in the 1980s. It will be really exciting for robotics if the electronics industry goes down that path.

While a mortal computer could work for some stuff, how are you going to train it? It seems like each computer would need to be trained on a large training set, which is going to be slow. If the computers are the same, then it seems like it is going to make it a lot faster to upload a trained model.

This reminds me so much of how "fuzzy logic" was going to COMPLETELY REVOLUTIONIZE EVERYTHING MAH DUDES back in the late 80s / early 90s.

Then as now, the question is still: "how does adding this pixie dust to boring old standard control logic make the product better?"

Sweet. A conversation with a toaster. This is exactly what everyone is clamoring for.