The fear of a nuclear fire that would consume Earth
bbc.com
bbc.com
Sounds like an overstretched analogy.
There is no such principle which says that just because a theoretical concern in one area turned out to not be a problem in practice, all theoretical concerns will not be a problem in practice.
It's not algorithms that stand in the way, it's crap computational capacity of our local physical reality.
Single human runs on 100W with algo and hardware optimized for more than million years. I think at some point we can do better with sand and electricity, but it's still far away.
And AI research is terrible with algos. Modern miracle GPT is plain old multilayer neural network with some forward shortcuts and some parts hard coded according to ad-hoc "attention" idea. And learning algorithm is more than half a century old. The reason GPT performs is just the RAM and compute.
We have every reason to be sceptical about AI apocalypse.
Just because 2 of the 10 chains restraining the cave troll are still intact does not mean it is time to make tea.
That makes perfect sense in 2023. Not so much in seventeen hundreds, or whenever people first started to attempting it.
> A handful of humans (a few hundred watts of compute) succeeded at devastating the world from 1939-1945.
I wouldn't really count it towards how powerful a person is. Literally billions of people amount to almost nothing their whole life. Wars were results of historical processes. People are nothing than a match under the barrel of gunpowder filled with economical and societal forces.
If there are such forces that could end us what's the difference if the match will be biological or silicon?
> Just because 2 of the 10 chains restraining the cave troll are still intact does not mean it is time to make tea.
Imagined cave troll.
I know it's not the same because AI research is closer to social sciences than to physics but still, there are parallels.
Huh: that linked Science piece is actually not correct. Its safety argument is that nuclear fission of silicon and oxygen, the most common elements in the Earth's crust, are endothermic. That's true. It's also backwards: the actual problem is that nuclear fusion of light elements is exothermic, and in fact self-sustaining under certain conditions (extreme mass density).
https://www.science.org/doi/10.1126/science.103.2687.760.a
The piece doesn't touch nuclear fusion at all. Was the expert not aware of it? He should be: he's discussing (astrophysical) novae as an analogy – and those are precisely runaway thermonuclear fusion detonations. (Didn't they know that in 1946?)
To be fair, he does clearly disclaim in that piece that he isn't a nuclear physicist. Still: it's remarkable that it's totally wrong, given the severity of the question it's attempting to answer (and the authoritative platform it's published in).