Having a tenuous grasp of how modern AI/DL networks work and an even more tenuous grasp of neuroscience, can I ask for a ELI5 of the above and a source?
Are you saying that Hinton is not a doomer and believes that AGI is inevitable very soon?
Are you saying that Hinton is not a doomer and believes that AGI is inevitable very soon?
I guess you know more than you let on but the tldr of modern AI is: massive GPGPU datacenters, automatic differentiation libraries, artificial neural architectures that scale by just stacking layers, and tera sized digital corpuses.
Here's a Hinton quote, as transcribed in https://www.computerworld.com/article/3695568/qa-googles-geo... which is presumably from his interview in https://www.youtube.com/watch?v=sitHS6UDMJc
--
"very recently, I’ve changed my mind a lot about the relationship between the brain and the kind of digital intelligence we’re developing. I used to think that the computer models we were developing weren’t as good as the brain. The aim was to see if you could understand more about the brain by seeing what it takes to improve the computer models.
Over the last few months, I’ve changed my mind completely, and I think probably the computer models are working in a completely different way than the brain. They’re using back propagation and I think the brain’s probably not. And a couple things have led me to that conclusion and one of them is the performance of GPT-4."
--
My understanding of the pre-joke-explaining "normie" interpretation of the tldr of AI was something like: 'the massive GPGPU datacenters are trying to substitute for the hardware of the brain but badly, inefficiently, and expensively. The automatic differentiation libraries and layer-stacking architectures are trying to substitute the amazing learning abilities of the neurons and synapses in the brain. The tera sized digital corpuses are used by AI in a statistically inefficient way compared to how much humans can learn from such little information.'
To me it seems that Hinton is now saying he's changed his mind about some of this interpretation, and that in particular it seems like he thinks the automatic differentiation libraries and layer stacking architectures are fundamentally more effective than what brains are doing with neurons and synapses, in a way that isn't just about clock cycles. This is funny to me because I don't think it's a common opinion on any side of the ai debates.
> Are you saying that Hinton is not a doomer
It depends what you mean by doomer, but I could speculate. I think he could be described as a dignified doomer with a stiff upper lip. He's 75 years old and I doubt he's holding out hope for a lot of life extension for himself. I don't think he expects to personally witness the real consequences of AGI even if AGI appears in his lifetime. I think he's pretty sure that AGI will eventually replace humanity as we know it.
> and believes that AGI is inevitable very soon?
maybe 'inevitable' is too strong but yes.