That's Hinton, Bengio and Sutskever.
Their voices should have a heavier weight than Andressen and other irrelevant with AI VCs with vested interests.
Although that is by no means the official position.
(Same can be argued for radio/tv/internet - "content" is what people talk about, but advertising is what moves money)
The Gutenberg Era lasted all the way from its invention to (I'd say) the proliferation of radio stations.
I'm not always that impressed by the discussion of AI or LLMs by engineers who undisputably have great things to say about the operation when they step outside their lane in predicting broader impacts or how the recursive content refinement is going to manifest over the next decade.
They aren't experts on what's encoded in the training data, as the last three years have made abundantly clear.
There's more to predicting the impact and evolution of technology than simply the mechanics of how it is being built today and will be built tomorrow (the area of expertise where they are more likely to be accurate).
And keep in mind that Hinton's alarm was sparked by the fact he was wrong about how the technology was developing, seeing a LLM explain a joke it had never seen before - a capability he specifically hadn't thought it would develop. So it was his failure to successfully predict how the tech would develop that caused him to go warning about how the tech might develop.
Maybe we should be taking those warnings with the grain of salt they are due coming from experts who were broadly wrong about what was going to be possible in the near future, let alone the far future. It took everyone by surprise - so there was no shame in being wrong. But these aren't exactly AI prophets with a stellar track record of prediction even if they have a stellar track record of research and development.
The idea that AGI poses an extinction risk like the notion of a chain atomic reaction igniting the atmosphere as opposed to posing risk more like multiple nation states pointing nukes at each other in a chain reaction of retaliation is borderline laughable.
The only way in which AGI poses risk is in its interactions with other systems and infrastructure, which is where knowledge of how an AGI is built is far less relevant than other sources of knowledge.
In an air gapped system no one interacts with AGI existing can and will never bring about any harm at all, and I would seriously doubt any self-respecting scientist would argue differently.
I trust AI VCs to make good arguments less than AI researchers.
Who do you think can better estimate the risk of engine fire in a Red Bull F1: the chief engineer or Max the driver? It is obviously the creator. And we are talking about invention safety here. VCs and other "tech gurus" cannot comprehend exactly how the system works. Actually the problem is that they think they know how it works when the people that created say there is no way of us knowing and they are black boxes.
Right. We should develop all arguments from commonly agreed, basic principles in every discussion. Or you could accept that some of these people have a better understanding, did put forth some arguments, and that it's your turn to rebuke those arguments, or point at arguments which do. Otherwise, you'll have to find somebody to trust.
It is good to get ahead of the curve, but there is also a lot of hype and overloaded terminology that is fueling the fear.
Also prompt engineering works for human too. It's called rhetoric, writing, persuasion, etc. Just because the intelligence of LLM's is different than humans doesn't mean it isn't a form of intelligence.
Speaking as a former cognitive neuroscientist, our current NN models are large, but simpler in design relative to biological brains. I personally suspect that matters, and that AI researchers will need more heterogeneous designs to make that qualitative leap.
I don't think more data would suddenly manifest all the nuance of human intelligence... that said we could just be one breakthrough away from discovering some principle or architecture, although I think that will be a theory first and not so much a large system that suddenly wakes up and has agency.
Hinton 732,799 https://scholar.google.com/citations?hl=en&user=JicYPdAAAAAJ
Bengio 730,391 https://scholar.google.com/citations?hl=en&user=kukA0LcAAAAJ
He 508,365 https://scholar.google.com/citations?hl=en&user=DhtAFkwAAAAJ
Sutskever 454,430 https://scholar.google.com/citations?hl=en&user=x04W_mMAAAAJ
Girshick 418,751 https://scholar.google.com/citations?hl=en&user=W8VIEZgAAAAJ
Zisserman 389,748 https://scholar.google.com/citations?hl=en&user=UZ5wscMAAAAJ
LeCun 332,027 https://scholar.google.com/citations?hl=en&user=WLN3QrAAAAAJ
On that basis Sutskever and He are both above Hinton, for example.
TBH It surprises me how controversial (1) is. The crux really is (2) ...
Can they? When plenty of groups are more interesting in exploiting the advances? When plenty of hackers will first try to work around the constraints - before even using the system as-is? When it's difficult even to define what AGI might test like or look like? When it might depend on a small detail or helper?
Did you accidentally click on a different article? It literally uses the word "cult" five times and does not demonstrate any knowledge whatsoever of the main arguments around AGI danger and AGI alignment.