What made Hinton into an AI doomer
wired.com
wired.com
His bold angle was that the automatic differentiability makes artificial optimization (training) so much more effective than any algorithm that biological neural systems might be using. According to wikipedia neural systems can do some backpropagation (https://en.wikipedia.org/wiki/Neural_backpropagation) but Hinton was suggesting that the artificial optimization methods can be so much more powerful.
This was funny to me because usually people allow biology the benefit of the doubt for the sake of argument, meekly begging to assume that artificial training might approach the biological abilities to within some order of magnitude, and submissively suggesting that one artificial neural parameter might be worth like 1/100 of a synapse or so if it would please the biologists. Then the AI researchers would say something like even if Moore's law diminishes we will still reach human power by 2050 or whatever. But not Hinton. He's like yeah the artificial training uses math power that goes beyond what our squishy neurotransmitters can muster, and if I were allowed to say AGI already exists then I'd say it, but I'm not allowed so instead I'll say AGI within five years.
Are you saying that Hinton is not a doomer and believes that AGI is inevitable very soon?
I guess you know more than you let on but the tldr of modern AI is: massive GPGPU datacenters, automatic differentiation libraries, artificial neural architectures that scale by just stacking layers, and tera sized digital corpuses.
Here's a Hinton quote, as transcribed in https://www.computerworld.com/article/3695568/qa-googles-geo... which is presumably from his interview in https://www.youtube.com/watch?v=sitHS6UDMJc
--
"very recently, I’ve changed my mind a lot about the relationship between the brain and the kind of digital intelligence we’re developing. I used to think that the computer models we were developing weren’t as good as the brain. The aim was to see if you could understand more about the brain by seeing what it takes to improve the computer models.
Over the last few months, I’ve changed my mind completely, and I think probably the computer models are working in a completely different way than the brain. They’re using back propagation and I think the brain’s probably not. And a couple things have led me to that conclusion and one of them is the performance of GPT-4."
--
My understanding of the pre-joke-explaining "normie" interpretation of the tldr of AI was something like: 'the massive GPGPU datacenters are trying to substitute for the hardware of the brain but badly, inefficiently, and expensively. The automatic differentiation libraries and layer-stacking architectures are trying to substitute the amazing learning abilities of the neurons and synapses in the brain. The tera sized digital corpuses are used by AI in a statistically inefficient way compared to how much humans can learn from such little information.'
To me it seems that Hinton is now saying he's changed his mind about some of this interpretation, and that in particular it seems like he thinks the automatic differentiation libraries and layer stacking architectures are fundamentally more effective than what brains are doing with neurons and synapses, in a way that isn't just about clock cycles. This is funny to me because I don't think it's a common opinion on any side of the ai debates.
> Are you saying that Hinton is not a doomer
It depends what you mean by doomer, but I could speculate. I think he could be described as a dignified doomer with a stiff upper lip. He's 75 years old and I doubt he's holding out hope for a lot of life extension for himself. I don't think he expects to personally witness the real consequences of AGI even if AGI appears in his lifetime. I think he's pretty sure that AGI will eventually replace humanity as we know it.
> and believes that AGI is inevitable very soon?
maybe 'inevitable' is too strong but yes.
Having said that, I assume that at some point in the future AI's problem-solving abilities will surpass those of humans. But there's still other issues to reckon with even then.
I think we should all be much more worried about what humans will do in response to AI than AI itself. Framing this all in terms of AI rather than AI-related policy distracts from the real issues in my opinion. I'm not worried about AI, I'm worried about people: what people will do with it and what rationalizations they will have for how they treat other people as a consequence.
He is not an AI doomer? Talking about the dangers of AI is not the same as being a doomer. He also has positive things to say about it.
A few tweets to qualify my claims
1. https://twitter.com/geoffreyhinton/status/163573945976432233...
2. https://twitter.com/geoffreyhinton/status/165457213374434508...
3. https://twitter.com/geoffreyhinton/status/165299357072121037...
And I see that even the article quotes anecdotes of him not being a "doomer". So I guess the headline is intentional and clickbait
But if "doomer" is allowed to include a Cambridge educated professor and Fellow of the Royal Society with a 'stiff upper lip' who finds himself surprised to realize that the caterpillar of humanity is creating the butterfly of AI and that "the caterpillar gets converted into a soup out of which the butterfly is created" as quoted in the first tweet thread that you listed, then maybe he could deserve the doomer label.
In another interview, in his response to 'why we should be scared' asked by a reporter he said that "as a friend of mine said, it’s as if some genetic engineers said, we’re going to improve grizzly bears; we’ve already improved them with an IQ of 65, and they can talk English now, and they’re very useful for all sorts of things, but we think we can improve the IQ to 210." Presumably this guy is understated enough to trust the listener to understand that he thinks that might reasonably scare someone.
> Talking about the dangers of AI is not the same as being a doomer
It is in fact the same thing as being a doomer if you're claiming that something unstoppable will lead to doom, in the same way that pointing out government coverups of lab leaks technically makes you a conspiracy theorist.
I mean, you could probably ask that question since at least 2016.
Computer hardware historically improves exponentially.
We should anticipate a system that thinks 100 times faster than a human in a few years or less. This is barely even speculative. Two orders of magnitude performance improvement has been achieved many, many times over in the history of computing. This is a very specific application and there is still probably relatively low hanging fruit for optimizing it with hardware and software customization. New compute-in-memory paradigms are coming soon.
The problem with hyperspeed GPT is that in an environment where companies and countries are competing, there is a strong incentive to give the AI very broad goals. Because having it wait a day for the humans to make decisions would mean 100 days of progress made by competitors (if it is at that speed). The human operators are effectively frozen relative to the activity of this speed of agent. They become speculators on the sidelines for the most part.
Also, you can't prevent people from instructing it to pursue its own selfish goals or having self-preservation or reproductive (copying code/model weights) goals. These will be open source models.
I think the only answer is that you ban the manufacture of AI hardware that exceeds a certain level of performance relative to humans. Also immediately start strongly discouraging people from simulating lifelike qualities in intelligent digital systems.
Recent discussion of Google's PaLM: https://news.ycombinator.com/item?id=30908941
The last thing I read about that was that nVidia hardware is better than TPUs now, those chips don't seem to be a particular competitive advantage (any more?) and all the AI startups seem to be using Azure anyway. Things change fast.
What a ridiculous charade. For years Google reacted to every OpenAI announcement or API launch by doing a blog post stating that they had already done it, their models were much better and they weren't releasing any evidence because their tech wasn't "safe" enough to make available to anyone, except, um, the benignly enlightened employees of Google themselves. And for years, like a fool, I believed them when they said their tech was better.
ChatGPT ripped the mask off, letting everyone see this supposed danger they had to be protected from for themselves. And it's laughable. Then Bard launched and we learned that Team Hinton's models weren't really better at all, that whole message was corporate misinformation or maybe self delusion or maybe both. Then it became "PaLM is definitely GPT-4 quality, we just aren't making it available yet". Uh huh.
Now we have this even more pathetic sight. Hinton, trying to set himself up as a classic public intellectual by extrapolating a trend into infinity until it reaches an end of the world that might happen "5-20" years from now, citing as evidence Proof By Snoop Dogg. He can go over there in the corner along with the "world hunger through overpopulation" and "New York under water by 2010" people. Having spent a lifetime chasing a goal he clearly never expected nor even wanted to achieve, he's now been lapped by his own former student who was busy actually making the AI revolution happen whilst Googlers were telling each other how virtuous they were.
And oh yes, these supposed great dangers. What are they, according to Hinton? Ourselves. Like all left wing purity spirallers, his biggest fear is that people like him lose their power over politics and information. We might believe things written by an AI instead of journalists. These people claim to live in fear of misinformation, but it's not credible. Where were they when US media was waging an industrial-scale misinformation campaign about non-existent Russian AI bots on Twitter? The ones who were supposedly secretly fielding GPT-4 quality bots to make Americans vote for Trump? So many claims, all fallen, and yet these brave anti-disinfo warriors so concerned about elections were nowhere to be found at the time.
Every time another round in this anti-progress, "let us smart people control things" salvo from Hinton appears, I breathe a deep sigh of relief that Team Sutskever exists and is doing so well. Left to their own devices Googlers would have happily sat around writing blog posts about how dangerous we all are forever. And I say that, sadly, as an ex Googler.
Where were they when US media was waging an industrial-scale misinformation campaign about non-existent Russian AI bots on Twitter? The ones who were supposedly secretly fielding GPT-4 quality bots to make Americans vote for Trump?
In that, simply put - these are straw-man claims that are being attacked here.
As to why - it's pretty obvious if one follows the event chronology of the broader story. Simply put: that the troll farms existed is a matter of forensic record, beyond dispute. But the narrative at the time was always in that formulation - "troll farms", i.e. essentially an MTurk operation. And yes, aspects of that narrative (specifically in regards to its actual effect on the election) were at the time speculative, and yeah, apparently lacking substance.
However - specific claims about them being "AI bots" didn't come until much later in the hype cycle, and were quite marginal (nowhere near "industrial-scale" as alleged above). And the claim about "secretly fielding GPT-4 quality bots" seems particularly strange, since GPT itself became known to the broader public until long after the election bots mania had arced and passed.
Here is a video compilation of just cases where MSNBC promoted just the Hamilton 68 dashboard, which supposedly tracked "Russian linked" bot and troll activity. It's 11 minutes of clips:
https://www.racket.news/p/eleven-minutes-of-media-falsehoods
These people couldn't stop talking about Russian bots. Listen for yourself and you'll see the claim is made regularly. Later it was discovered that these hundreds of accounts were all ordinary westerners, and the Hamilton 68 people admitted that they'd never really had a reason to think they were Russian. That's just one TV channel, referring to one source of misinformation! There are far far more such clips available.
This sort of thing was totally normal outside the media too. Look at how many papers have come from academics writing about this supposed widespread problem:
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=%22s...
Over 14,000 papers on bots that don't really exist!
And that's just a tiny fraction of the misinformation pumped into society regularly. Again, if Hinton actually cared about misinformation, he'd have cared about that. He doesn't. Like everyone else claiming to care, he's simply a bitter partisan, hence his comment about Biden at the end of the interview.
My point is that these claims all revolved around the idea that the internet was being flooded with bots that spoke and thought just like Americans, so similarly that people couldn't tell they weren't human and would even be highly persuaded by them. Yet the tech to do that didn't exist at the time. People who observed this fact were of course attacked as spreading misinformation themselves, which is why the whole concept is just a joke idea.
Right. Long and short of it is, even though a lot of people were talking about "bots" at the time, they weren't really thinking too much about what that meant, technically. Or even whether these "bots" were actually just literal rooms of people typing garbage from a script (by far the likeliest explanation - save for those which have since been verified as "authentic" right wing troll accounts).
Either way -- none of this was ever an "AI risks" story. That is -- nothing for people Hinton to be concerned about, let alone to have to answer to.
But hey, you can remember it any way you want. It's a free country after all.